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Introduction 


This fourth edition of the Global Inventory of 
Regional and National Qualifications Frameworks 
arrives at a crucial moment in the relatively short 
history of qualifications frameworks as tools 

within the wider reforms of education and training 
systems. Numbers of frameworks, national and 
regional, remain stable, while implementation of 
most frameworks has deepened and widened since 
2017 At the same time, frameworks are evolving 

in a context of everfaster technological, social 

and economic changes. These include intensified 
globalisation and internationalisation of labour 
markets, the advance of digital technologies, and 
migration. Much of this change has a direct bearing 
on the purposes and function of frameworks, 
notably the search for systems which compare skills 
and qualifications internationally, the emergence of 
digital credentials, and the development of methods 
and tools to recognise the skills of migrants 

and refugees, especially where those skills are 
undocumented or not formally certificated. 


We observe first, in national implementation, 

that learning outcomes underpin most European 
education and training systems, as well as those 

in many other industrialised countries around 

the world. Further, they are making significant 
inroads in developing or transition countries, from 
the European Neighbourhood of Eastern Europe 
to the Southern Mediterranean, Africa, Asia and 
the Pacific. Internationally, regional qualifications 
frameworks and the World Reference Levels — an 
emerging global tool to describe and compare an 
individual's skills and qualifications — use learning 
outcomes as their conceptual basis and common 
language. The World Reference Levels have 
advanced significantly since 2017, and will shortly 
enter general use. The ASEAN Qualifications 
Reference Framework becomes the second world 
region framework to be operationalised, joining the 
European Qualifications Framework. But the story 
is not one of universal progress. Some countries 
are hampered at governance and institutional levels 
by weak coordination of stakeholders, insufficient 
cooperation between different sectors, and an 
absence of adequate regulatory oversight. They are 
also held back in implementation by poor quality 

of training provision, insufficient opportunity for 
teacher continuing professional development, and 
uneven application of outcomes in curricula and 
qualifications. 


At the same time, entirely new systems and 
symbols of accreditation and credentialing are 
gaining ground in the day-to-day negotiation 
between learners, workers and employers, resulting 
in the rise of digital credentials, a term covering 

a range of methods and tools which capture 

and recognise learning outcomes. Countries are 
increasingly confronting the issue of how to link 
frameworks more directly to learners and workers, 
by examining how they know about skills and 
qualifications via information systems and tools and 
career guidance, and how they can acquire them, 
through learning and teaching. Here, again, we 
observe the centrality of outcomes, as curricula and 
teaching reforms are increasingly influenced by this 
underpinning concept in qualifications. 


Perhaps the most pressing, and certainly the most 
rapidly evolving challenge qualifications frameworks 
must face, is that of digitisation of economies and 
societies. Digitisation has spread further and faster 
than any preceding innovation, simultaneously 
exposing and expanding major challenges to 
economic growth and social cohesion. Digitisation 
deserves attention in its own right because it has 
emerged as arguably the most important driving 
force in the economy today. It not only disrupts 
established business practices but it has a sizable 
impact on labour markets by creating new jobs and 
destroying others, and by requiring people to update 
their skills. 


These changes have raised interest in lifelong 
learning by its inclusion as a central feature of 

the 2030 Agenda for Sustainable Development, 
where in Goal 4 in particular, UN member states 
have committed to ensure inclusive and equitable 
quality education and promote lifelong learning 
opportunities for all (UN, 2015). The recent release 
of the report by the ILO Global Commission on 
the Future of Work has also given high attention 
to lifelong learning. The report calls for a human 
centred agenda for the future of work that ‘means 
investing in people's capabilities, enabling them to 
acquire skills, reskill and upskill and supporting them 
through the various transitions they will face over 
their life course’ (ILO, 2019, p. 24). 


Actors and stakeholders engaged in developing 
qualifications frameworks are alert to these various 
challenges, and are responding by adapting and 
updating their frameworks and associated tools 
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and systems, notably by focusing on transparent, 
multi-level governance; decentralised and flexible 
cooperation mechanism; and inclusive and equitable 
social and technological standards. Governments, 
international organisations, and, encouragingly, 

the private sector are all engaged in these efforts 
to develop qualifications frameworks to meet 
these challenges. The participation of all sectors is 
necessary to ‘repurpose our education systems for 
lifelong learning, for resilience and for emotional 
and social intelligence in the face of uncertainty 
and change, as UN Deputy Secretary General 
Amina Mohammed put it'. The contribution of 
regional and national qualifications frameworks to 
this repurposing should not be underestimated, 
because when it comes to proving skills in order to 
access or to allocate work, fundamental questions 
of trust between individuals and institutions 

and, increasingly, across borders, must still be 
addressed. Ouality-assured qualifications, that 

is qualifications which meet conditions of labour 
market need and learner relevance, as determined 
by stakeholder engagement, remain the best 
representation of an individual's skills. Qualifications 
frameworks are instruments not only for collating 
and comparing qualifications, but also for activating 
them as enablers of systemic — and societal — 
diversity and development. 


Scope of the inventory 


The present edition of the Global Inventory of 
Regional and National Qualifications Frameworks 
maintains the role established by preceding 
editions, gathering knowledge and insight from the 
international range of contexts in which regional 
and national qualifications frameworks operate. In 
continuing this observatory function, the inventory 
highlights progress and areas for improvement in 
the context of emerging trends and policy priorities. 


As with the previous edition, this volume draws on 
inventories from four organisations: the European 
Centre for the Development of Vocational Training 
(Cedefop), the European Training Foundation (ETF), 
the United Nations Educational, Scientific and 
Cultural Organisation (UNESCO) and the UNESCO 
Institute for Lifelong Learning (UIL). As ever, their 
observations are based on the close working 
relationships with country officials, experts and 
practitioners that create a vital two-way flow of 
news, data and documentation. It also draws on 
extensive international research in the development 
of digital qualifications, accessing and acquiring 
skills and qualifications, the proliferation of learning- 


1 ~~ www.un.org/press/en/2019/dsgsm1252.doc.htm 


outcomes-based management within wider reform 
efforts, and approaches to the recognition of prior 
learning, both in general and for migrants and 
refugees in particular. 


Volume Il comprises more than 100 national 
qualifications framework country chapters, from 

all the continents, plus seven world regions 
qualifications frameworks chapters. The country 
chapters follow a similar, comparable format, 
describing the characteristics and analysing the 
development and implementation of the respective 
national frameworks. They provide the educational, 
social, economic and policy context which shape 
countries’ national qualifications frameworks; 
assess the application of learning outcomes in the 
national education and training system; Summarise 
the governance, stakeholder and legislative 
arrangements countries have adopted; examine the 
links between national qualifications frameworks 
and associated policies, instruments and services, 
including validation of non-formal learning; and 
conclude with an assessment of impact, Successes 
and challenges. 


Thematic focus 


The current edition’s thematic focus is evidence of 
the degree to which three core areas are dominating 
the direction of reforming policies; digitisation, 
recognition of prior learning, and outcomes-based 
approaches to managing education and training. 
The chapters follow a structure reflecting these 
three areas. We begin with digital dimensions in 
methodologies for constructing and comparing 
qualifications, before moving on to a wide-ranging 
discussion of the development and impact of 
learning outcomes as part of a management-by- 
outcomes imperative in public institutions and 
systems, and then examine recognition, validation 
and accreditation of prior learning through the 
perspectives of international policy trends and 
specific case studies, as well as the experience of 
migrants and refugees. The volume concludes with 
an exploration of the benefits which can be gained 
from forging closer links between qualifications 
frameworks as policy tools, and the practitioners 
whose work makes them meaningful and valuable 
to individuals and society as a whole. 


Digital dimensions of comparison and 

change in qualifications 

In Chapter 1's examination of world reference 
levels for lifelong learning, John Hart and Borhene 
Chakroun pick up the story from the final w of 

the previous edition (by Chakroun and Katerina 
Ananiadou). In response to the internationalisation 
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of education and training systems and labour 
markets, including evergreater mobility within 

both spheres, the International Congress on TVET 
held in Shanghai in 2012 witnessed a commitment 
by UNESCO to develop World Reference Levels 

to function as internationally-viable guidelines 

on quality assurance for the recognition of 
qualifications. Chakroun and Hart establish the 
background to this effort, and explain how the World 
Reference Levels tool ‘translates’ sets of learning 
outcomes into standardised, globally recognisable 
reports using a digital app. Despite the great variety 
of ways in which job requirements and education 
and training programmes are described, the authors 
identify a set of pervasive themes, which can be 
characterised as three questions posed from the 
learner's perspective: What am | accountable for? 
What must | be able to demonstrate or apply? And 
what must | be capable of responding to? In other 
words, these questions exemplify the competences 
attaching to Elements of Progression within 

the World Reference Levels — accountabilities, 
capacities and contingencies, respectively. 


Chakroun and James Keevy consider digital 
credentials and interoperability in Chapter 2, ina 
discussion extracted from a previous UNESCO 
report looking at the rise of new credentialing 
methods. Employers, learners, and parents are 
increasingly expressing dissatisfaction with the 
degree as a ‘gold standard’ for labour market 
access. New ‘micro’ credentials offer cheaper, 
faster routes, and potentially greater returns 

on the investment required to attain them. 

Micro credentials, such as digital badges, are 
distinguished from traditional credentials by a 
number of characteristics, including ‘stacking’ 
(i.e. sequential, asynchronous accumulation) and 
machine readability, although these remain subject 
to trust and quality assurance issues. UNESCO's 
report proposes an ‘ecosystem’ of credentials 
based on use, provision, awarding, quality 
assurance evaluation, verification and governance, 
and describes various ‘architectures’ of tools and 
technology currently in use. 


Concepts and impacts of learning outcomes 

Jens Bjernavold, in Chapter 3, explores the growing 
influence of the learning outcomes approach, 

and its active use as a reform tool within policy, 
institutional management, and pedagogy. The 
author notes that the use of the learning outcomes 
approach in these areas is associated with high 
ambitions and aspirations. Drawing on the latest 
research, he discusses how far these aspirations 
and ambitions have been met. One concern is 
identifying the extent to which a shift to learning 
outcomes at political and administrative level 


actually influences teaching and learning processes. 
The author argues that while the learning outcomes 
approach in recent years has started to influence 
qualifications systems, standards and curricula, its 
impact on the teaching and learning processes is 
still limited. In the longer term, the author believes, 
the transformative potential of learning outcomes 
depends on the way they are used to facilitate 
more learnercentred approaches in education and 
training. 


Recognition, validation and accreditation of prior 
learning: What to count? Whom to include? 

The next two chapters cover critical aspects of the 
recognition, validation and accreditation of prior 
learning (RVA), also termed validation of non- 
formal and informal learning (VNFIL), or recognition 
of prior learning (RPL). In Chapter 4, Ernesto 
Villalba-Garcia and Borhene Chakroun look at RVA 
models and monitoring, before presenting four 
short case studies. In Chapter 5, Ruud Duvekot 
and Raul Valdés-Cotera consider the experiences 
of migrants and refugees, and the struggle for 
inclusive policies and practices. Villalba-Garcia and 
Chakroun’s discussion is framed by the centrality 
of RVA in the UN Sustainable Development 

Goals, specifically Goal 4, ‘to ensure inclusive and 
equitable quality education and promote lifelong 
learning opportunities for all’. RVA contributes 

by enabling people to adapt to a flexible and 
fast-changing labour market, and can reduce 
‘qualifications deficit’ amongst adults. It serves as a 
holistic assessment of an individual’s capability at a 
specific point in time, but it is conducted differently 
in different countries, depending on a large number 
of variables. The authors set out the main features 
of validation in four mini case studies, from France, 
South Africa, Portugal and the United States. They 
go on to suggest the main dimensions of RVA 
monitoring and evaluation, in the shape of four 
questions: Are those most in need of RVA of prior 
earning able to access it? Is the approach to RVA in 
any given country of sufficient quality, with suitable 
indicators identified? Are the outputs — in the form 
of certification — right in terms of quality? And are 
the outcomes fit for purpose for individuals and 

for employers? The case studies present valuable 
details of approaches to monitoring and evaluation 
of RVA for decision-making, and show the 
importance of precise definitions of what is meant 
by RVA and how its outcomes are to be measured. 


From the perspective of those who are deemed 
‘newcomers’ in a society —i.e., migrants and 
refugees — social inclusion is inextricably linked 

to learning and career opportunities. RVA plays 

a significant part in facilitating integration, and in 
Chapter 5 Duvekot and Valdés-Cotera demonstrate 
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how appropriate interventions by and with key 
stakeholders in the RVA process lead to benefit for 
all. People move for different reasons, sometimes 
by choice and sometimes not. Whatever the causes, 
the numbers are growing. Close to a quarter of 

a billion people do not live in their home country, 
according to the UN's 2017 Global Migration Report. 
When newcomers are learning or working (or 
both), they and their host communities benefit. 
Putting RVA at the core of flexible pathways helps 
to make that possible. High levels of migration put 
pressure on education and training systems, and 
limit the ability of those systems to enable the 
access to learning that people need to ‘shape their 
own destinies’. Migrants and refugees often find 
the doors of receiving countries’ formal education 
and training systems closed to them. But several 
countries are developing legal, regulatory and civil 
society initiatives to promote skills development 
and labour market access, thus enabling better 
integration. 


Understanding the benefits of skills and 
qualifications 

To deliver benefits to learners and employers, 
national qualifications frameworks need to be 


connected to system elements that allow people 
to acquire skills and qualifications. National 
qualifications frameworks are now in place in 
many of the ETF's partner countries in the EU 
neighbourhood, but reforms are needed in system 
elements to establish meaningful connections 
between people and the benefits that a national 
qualifications framework can bring. After all, 
learners, workers and employers are people, not 
merely abstract concepts. The issue is to identify 
how people know about and acquire skills and 
qualifications, and to treat these two related, but 
not identical, markers of human capital in the 
same way. A forthcoming ETF study has set out 
to understand the enablers and barriers of such a 
process, which is summarised in Chapter 6, 
co-written by Michael Graham, Arjen Deij, Mirjam 
de Jong, Jolien van Uden, Carmo Gomes and 
Eduarda Castel Branco. The study is guided by 
three core questions: How do people know about 
skills and qualifications? What types of skills and 
qualifications do they need? And how can people be 
supported to acquire them? The authors consider 
how countries can enable people to access and 
acquire skills and qualifications, by looking at system 
elements such as career guidance, curricula, and 
teaching and learning. 
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CHAPTER 1. 


WORLD REFERENCE LEVELS FOR LIFELONG 
LEARNING: A TOOL FOR COMPARISON AND 
RECOGNITION OF LEARNING OUTCOMES 


John Hart, independent consultant, and Borhene Chakroun, UNESCO 


Summary 


Following the recommendation of the the 

Third International Congress on Technical and 
Vocational Education and Training (TVET), held in 
2012, UNESCO has been working with experts, 
international partners and regional organisations 
overseeing regional qualifications frameworks to 
develop a set of World Reference Levels (WRLs) to 
facilitate international recognition of qualifications. 
This chapter describes the process by which a tool 
has been developed to achieve this aim. The WRL 
tool takes the form of a digital levelling instrument 
(the WRL app) which can turn the specification or 
description of a qualification, credential, or other 
set of outcomes into a standardised analytic profile 
and report capable of supporting international 
comparisons and recognitions. The chapter also 
describes the plans for piloting the WRL tool to 
ensure that it can support better, fairer and more 
transparent recognition of skills and qualifications 
within and across borders. 


1.1 Introduction and background 


Since 2014, several reports and articles (Chakroun, 
2013; Keevy and Chakroun, 2015; Chakroun and 
Ananiadou, 2017) have reported on the progress 

in developing World Reference Levels (VWRLs) 

for learning outcomes. The work on WRLs was 
undertaken in response to a recommendation of the 
Third International Congress on TVET in Shanghai in 
2012. The outcomes of this congress, known as the 
Shanghai Consensus, contained nine far-reaching 
recommendations aimed at improving the role and 
quality of TVET around the world, and in one of 
these, the United Nations Educational, Scientific and 
Cultural Organisation (UNESCO) was called upon ‘to 
identify a set of world reference levels, to facilitate 
the international comparison and recognition of 
TVET qualifications’ (UNESCO, 2012, p. 6). 


In the decades preceding and following the 
Shanghai Consensus, three global trends have 
combined to give urgency to that call: continuing 


growth in migration around the world, the 
increasing globalisation of the labour market, and 
the internationalisation of education and training. 
Recent figures from the United Nations (UN, 

2017) estimate that 258 million people are living 

in a country other than their country of birth, an 
increase of 49% since 2000. People migrate for 
many reasons: to seek or take up work; to study; 
to be closer to family or friends; to be part of a 
way of life that is more in tune with their wishes 
or values; or to escape oppression, persecution 
and terror. For most of these individuals, whatever 
their reason for migrating, it is important to have 
their qualifications (in whatever form they take) 
recognised to give them access to employment or 
to education and training. This recognition may be 
within a national education system, by employers 
and/or by an occupational sector. The volume of 
migration has created a need for mechanisms and 
agreements to assist both those making claims for 
recognition and those responding to these claims by 
giving employment or granting admission to further 
learning. 


The recently adopted Global Compact on Migration 
for Safe, Orderly and Regular Migration was 
endorsed by the UN Member States on 13 July 
2018 and adopted in December 2018. Recognising 
the need for a Global Skills Partnership, Objective 
18 is dedicated to the issue of investing in skills 
development and facilitating the recognition of 
skills, qualifications and competences. It calls 
upon Member States to ‘[b]uild global skills 
partnerships amongst countries that strengthen 
training capacities of national authorities and 
relevant stakeholders, including the private sector 
and trade unions, and foster skills development 

of workers in countries of origin and migrants in 
countries of destination with a view to preparing 
trainees for employability in the labour markets of 
all participating countries’ (UNGC, 2018, pp. 25-26). 
In this way, It responds to Goals 4 and 8 of the 
UN Sustainable Development Goals, which call 
for countries to ‘ensure inclusive and equitable 
quality education and promote lifelong learning 
opportunities for all’ (Goal 4) and to ‘promote 


sustained, inclusive and sustainable economic 
growth, full and productive employment and decent 
work for all’ (Goal 8). 


In today's world, some migration comes about 
because employers are taking advantage of the 
relative ease with which they can now reach over 
borders to find individuals with the capabilities they 
need. However, the wider the spread of countries 
from which they want to recruit, the more national 
qualifications systems they need to be familiar with. 
This internationalised labour market has contributed 
to the need for a new way to compare qualifications, 
credentials and other outcomes of learning from 
different systems. 


Learning across boundaries has become a global 
phenomenon. Today, not only people and jobs 

but also study programmes and education and 
training institutions are increasingly internationally 
mobile. This increased mobility calls for fair and 
valid recognition of learning outcomes at national, 
regional and international levels. However, migrants 
at all skills levels, particularly medium and lower, 
are often vulnerable to labour exploitation and 

lack of protection. Labour market integration and 
career development are key factors in improving 
productivity, social cohesion and sustainability. 
Recognised skills and qualifications can be an 
asset for migrants’ integration, and in order to reap 
the potential benefits and advantages, migrants’ 
learning and qualifications achieved before and 
after migration have to be recognised, valued and 
further developed in their countries of destination 
and origin. 


The work on WRLs is also affected by the growing 
digitisation trends. Future developments will need 
to take into consideration at least two areas: (i) 
the emergence of new types of digital credentials, 
including open and online badges; and (ii) the use 
of digital technologies to improve the quality and 
transparency of the recognition process itself 
(Keevy and Chakroun, 2018). 


1.2 Progress in developing WRLs 


Numerous processes for registering, classifying, 
comparing, matching and referencing qualifications 
and credentials have been, and are being, created 
to respond to the need to compare and recognise 
qualifications across international boundaries. 
UNESCO has now developed a WRL tool that can 
supplement and support these processes. The 

tool is capable of helping to meet the needs of 
individuals looking for recognition and of recruiters 
or gatekeepers (such as credential evaluators 


and national recognition agencies) looking to give 
recognition. 


Although the impetus for the WRLs was strongly 
vocational, as contained in the Shanghai Consensus 
Recommendation (UNESCO, 2012), the decision 
was made not to limit the work only to TVET 
levels. This decision was based on the argument 
that level descriptors are generally used for all 
levels and sectors of education and training in 

a lifelong learning perspective, and, hence, the 
WRLs would be more widely applicable. The WRLs 
will offer a lingua franca that can be used not only 
with technical and vocational qualifications and 
credentials, but also with professional, general 

and academic qualifications, credentials and less 
formal sets of learning outcomes, and with job 
specifications and entry requirements for learning 
programmes. 


The WRL tool works by translating sets of learning 
outcome statements? into standardised reports, 
using a conceptual structure that should be 
recognisable around the world. It will work with all 
kinds of qualifications and credentials’, and also 
with other significant sets of outcome statements, 
such as level descriptors, occupational standards, 
learning programmes and requirements for 
admission to learning programmes, job descriptions 
and person specifications for all kinds and levels 
of work roles. An important advantage of the 

WRL tool is that because it focuses on translation 
into the lingua franca, it can be used without any 
requirement to change or realign the qualifications, 
credentials or other sets of outcomes that it 
translates and reports on. As illustrated in Figure 1.1, 
this means that any kind of qualification, credential 
or other set of outcomes, can be processed by the 
WRL tool without requiring any change in format 
or content, to produce a standardised profile and 
report, capable of being compared with any other 
WRL profile or report. 


The WRL tool has been developed with the support 
of an international expert group established by 


2 The term ‘outcome statement’ is used here to denote a 
ormal description of what an individual is required to be 
able to do, including demonstrating or applying knowl- 
edge and understanding, for a particular purpose. It may 
elate to the assessable outcomes of a qualification or 
credential, or to the essential capabilities or competences 
in a job specification, or to the conditions of entry to a 
programme of learning. 

3 Increasingly, these may take the form of online provision 
eading to some kind of record of achievement and might 
be labelled a digital badge, a nano-degree, a micro- 
certification, a web badge, a mini-degree or an open 
badge. 
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Figure 1-1. The WRL process 
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Source: Authors. 


UNESCO in 2015. The views of the expert group 
were shaped into the following principles, which 
were used in creating the WRLs and the WRL tool. 


1. The main purpose of the WRLs should be 
to assist actors across the world to make 
comparisons of the outcomes of lifelong learning 
and reach agreements on the recognition of 
qualifications and credentials. 

2. The WRLs should be capable of being used in 
conjunction with qualifications frameworks and 
other frameworks of outcomes from around the 
world — frameworks with different purposes, 

different numbers and forms of levels, and 

different approaches to defining levels. 

3. The WRLs should take the form of a tool, 

building on, but not replicating or replacing, 

the work of existing national and regional 

qualifications frameworks. 

4. The WARL tool itself should not act as a 

qualifications framework. 

5. The WRLs should describe a range of elements 
of capability that should be derived from, and 
be compatible with, the contents of level 
descriptors used in qualifications frameworks 
and related structures. 

6. The function of the WRL tool should be to 
turn the user's specification or description of a 
qualification, credential, or other set of outcomes 
into a standardised analytic report capable 
of supporting international comparisons and 
recognitions. 

7, These standardised reports should combine 
factual information, professional judgements and 
the evidence for these judgements. 

8. They should be relatively detailed in form, but 
easy to generate. 


~ WRL 


Sy clits 
WRL 
Directory 


9. The primary means of delivering the VWRLs 
should be digital, but an analogue (paperbased) 
version should be available. 


REPORT 


The result of this work is aWRL tool in two forms: 

a digital levelling instrument (the WRL app) and a 
back-up paperbased instrument. The WRL tool is 
used to create a graphical profile that translates 

any set of outcomes into WRL terms, and a more 
detailed report for any qualification or credential 
that has been awarded on the basis of quality- 
assured assessment. The profile and report are 
based on eight WRL stages of progression and 
eleven WRL elements of capability backed by a WRL 
directory of clearly defined and cross-referenced 
concepts. These three components of the WRLs 
(stages, elements and directory) are designed 

to support clear communication between actors 
from different countries and different sectors with 

a shared interest in extending the understanding 
and recognition of the outcomes of learning. The 
following sections present the three components of 
the WRLs and the digital tool. The paper ends with a 
conclusion regarding the next steps for piloting the 
WRL tool in different contexts. 


1.3 The WRL tool 


At a time when national agencies, awarding bodies, 
education institutions and employers have to deal 
with a growing range and diversity of qualifications 
and credentials from different systems, the WRL 
tool offers an accessible way to translate the many 
frameworks of qualifications, credentials and 

other set of outcomes from all sectors of learning 
and work into shared and exchangeable terms. It 
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does this by breaking down the large questions of 
equivalence and recognition into smaller elements 
and indicators, and guiding the user through the 
process of creating a WRL profile and report based 
on these elements and indicators, while allowing 
the user to keep control of both inputs and outputs. 
These outputs will then be available for scrutiny and 
acceptance, further investigation or rejection by the 
organisations or institutions responsible for making 
decisions on recognition. 


1.3.1 The levelling 

The WRL tool builds on some of the common 
characteristics of outcomes-based frameworks for 
classifying qualifications, credentials and other sets 
of outcomes. At first sight, it may appear to be an 
ambitious qualifications framework. However, in a 
number of important general and specific respects, 
the WRL tool is significantly different from most 
frameworks for qualifications, credentials and 
competences. 


With the WRLs, the aim is to produce a spectrum of 
information about how qualifications, credentials or 
other sets of outcome statements relate to a range 
of elements of capability. Different combinations 

of these elements define what is required to carry 
out specific roles in work or study. Each element 
will be recorded as being at a specific stage of 
progression or be flagged as ‘not fully relevant’ 

to the qualification, credential or other set of 
outcomes that is being profiled. This will make 
comparisons of specific qualifications, credentials 
and competences accessible, informative and more 
detailed than is usual when comparisons are based 
only on framework levels. The need for best fit 

will be reduced and there will be no requirement 
to average out the findings. This approach will 

give a relatively detailed and wide-ranging picture 
of the qualification, credential or competence in 
standardised terms that can be understood across 
the community of WRL users and easily related to 
any number of frameworks with any number 

of levels. 


Research into a wide range of frameworks 
of different kinds* suggested that, in order to 
strengthen their applicability, the WWRLs should 


4 These included national, regional and sectoral qualification 
frameworks, competence frameworks, knowledge and 
skills frameworks for specific occupations, job evaluation 
frameworks, occupational information/career guidance 
frameworks, the European Dictionary of Skills and Com- 
petences (DISCO), the European classification of Skills, 
Competences, Occupations and Qualifications (ESCO) 
and the International Standard Classification of Education 
(ISCED) and of Occupations (ISCO). 


have a restricted number of levels and four broad 
levels were identified. These are briefly described 
below. 


e Level A ranges from the essential demands of 
modern society, learning and work (functional 
literacy, numeracy, use of information and 
communication technology, general knowledge) 
to the capabilities associated with basic studies 
and simple work roles. 

e Level B ranges in academic terms from the 

kinds of outcomes typically set for the end of 

compulsory education to the kind of outcomes 
typically associated with entry to tertiary or 
higher education studies. In work terms, it 
ranges from the capabilities required to carry out 
relatively independent, but routine, work roles to 
illed work and supervisory roles. 

e Level C is characterised by the capabilities 

associated with the first cycle of tertiary or 

higher education or the work roles of para- 
professionals, junior professionals, specialists 
and managers. 

e Level D is characterised by advanced intellectual 
and occupational capabilities. It includes 
outcomes associated with the second and third 
cycles of higher education, and the activities 
and responsibilities of independent specialists, 
technologists, analysts, and executives with 
extensive and/or strategic responsibilities. 


n 


Because these are deep levels, progression within 
the levels has been built into the structure, with 
lower and higher stages of development identified 
at each level. These are the eight WRL stages of 
progression, labelled A1 and A2, B1 and B2, C1 and 
C2 and D1 and D2. Each pair of stages shares forms 
of capability that are characteristic for the level, but 
each stage also identifies crucial differences in the 
quality, scope or conditions that apply to the shared 
characteristics and define the stage. 


1.3.2 Critical factors in defining levels 

The WRL tool creates a graphical profile and a 
structured report on qualifications, credentials and 
other sets of outcomes based on 11 elements of 
capability. These elements are derived from the 
concepts that combine to define levels and show 
where progression can take place in different kinds 
of framework related to qualifications, learning and 
employment. They are referred to here as ‘critical 
factors’. They were initially identified by analysing 
the level descriptors of a range of qualifications 
frameworks to identify the most commonly used 
factors, and these findings were then compared 
with a range of other related structures to ensure 
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breadth of applicability. More than 20 factors were 
identified in the first set of analyses®. 


A second round of analyses was carried out, 
focusing more closely on the nature of level-to-level 
progression in the level descriptors of the sample 
frameworks. As a result, the number of critical 
factors could be reduced from 20 to 11. These were: 


e nature and scope of academic and occupational 
activities; 

e knowledge and know-how; 

e skills and procedures; 

e discrete communication skills; 

e accessing and using data; 

e problem-solving and research; 

e type and degree of responsibility for these 
activities; 

e working with others; 

¢ monitoring performance and learning to maintain 
or improve quality; 

e action relating to values; 

e the context of activities. 


These factors were the bases on which the WRL 
elements of capability were developed. 


1.3.3 An organising structure for progression 

in the WRLs 

The order in which the critical factors appear in 
frameworks and related structures varies a great 
deal, depending on whether the descriptors 

are more oriented towards learning or towards 
performance: frameworks that are oriented towards 
learning usually start with knowledge; frameworks 
that are oriented towards performance usually 
start with activities or contexts. Meeting the aims 
of the WRLs required the creation of a neutral 
arrangement that reflects a more generic way of 
thinking about capability and progression. The next 


5 In qualifications frameworks, the factors are usually 
grouped under broad headings such as ‘knowledge’, 
‘skills’, ‘application’, ‘practice’, ‘role’, ‘context’, ‘responsibil- 
ity’, ‘autonomy’, ‘leadership’, ‘social skills’ and ‘life skills’. 
The formula ‘knowledge, skills and competences’ is often 
used to represent this diverse range of headings, but 
this formulation hides the extent to which the headings 
can, in practice, differ in both scope and meaning. In 
current frameworks, the groupings are variously referred 
to as ‘characteristics’, ‘criteria’, ‘domains’, ‘indicators’, 
‘categories’ and ‘strands’. The number of groupings can 
range from two to five, and some groupings may then be 
subdivided into two or three sub-groupings. Within each 
grouping or sub-grouping, there is usually a number of 
outcome statements, each isolating a critical factor, at 
each level. Specific factors (such as depth of knowledge, 
breadth of knowledge, type of skill, communication skills, 
leadership, responsibility for quality) can appear under 
different headings in different frameworks, or even under 
different headings in a single framework. 


step, therefore, was to place these common factors 
into a new organising structure that brought these 
different perspectives together. 


To accord with the purposes and principles of the 
WRLs, any taxonomic structure would have to 
be applicable to all kinds of academic, social and 
occupational outcomes achieved through formal, 
non-formal and informal learning. It would also 
have to link clearly to common academic and 
occupational levels, or stages of progression, in a 
form that could be shown to link to users’ everyday 
concerns. Given the anticipated range of users, 
the taxonomic structure would have to be flexible 
enough to adapt to sectoral needs, but robust 
enough to maintain a common identity across 
sectors. 


Following the recommendations of Keevy and 
Chakroun (2015), a review of classic taxonomies 
of learning and outcomes was carried out; this 
included work by Bloom (1956), Anderson and 
Krathwohl (2001), Biggs and Collis (1982) and 
Dreyfus and Dreyfus (1986). Work by Marzano and 
Kendall (2007) and Denning and Flores (2016) was 
also considered. These reviews showed that while 
these taxonomies were useful in offering evidence 
about the nature of certain kinds of progression in 
the acquisition of different kinds of understanding 
and skill, none of them would meet all the needs of 
the WRLs. 


In broad terms, It can be said that these 
taxonomies are focused on understanding 
progression in learning with the aim of improving 
teaching/training and assessing. However, VWWRLs 
need to capture and report on the broad range 

of capabilities that are the focus of negotiations 
on recognition of qualifications, credentials and 
other forms of learning at different levels to give 
access to both further learning and employment. In 
order to make the reports produced by the WRLs 
recognisable to teachers, academics, trainers, 
managers and employers, an organising structure 
is required that gives equal place to outcomes 

of learning, performance and experience, and 
classifies the WRL elements in a way that reflects 
ways of thinking about qualifications as a basis 
for progression in both the educational world and 
the world of work. The most immediate sources 
for this kind of focus are prospectuses for school, 
college and university programmes, and job and 
person specifications for posts at different levels in 
different sectors. 


An initial review of relevant documents and 
websites showed significant differences in formats 
for describing programmes and posts. However, 
three pervasive themes or perspectives emerged 
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that could be used to organise the 11 critical factors 
identified from common usage in qualification and 
other outcomes-based frameworks into the WRL 
elements of progression. These might be said to 


focus on what individuals should expect to (i) be 
accountable for; (ii) be able to demonstrate or apply; 
and (iii) be capable of responding to. This is shown 
in Box 1.1. 


Box 1-1. Element clusters as they may appear in work and study specifications 


Programme/qualification specifications 


1. Nature of the course of learning — structure and level 
2. Expected or required outcomes — expressed as skills and knowledge 
3. General information about potential learning and assessment arrangements 


Programme/course entry requirements 


1. Experience in/capacity to undertake a specific type of study/work* 
2. Evidence of specific skills, knowledge and/or experience* 


3. Capacity to learn in certain contexts 


Job adverts 


1. Nature of the employer's business and nature of the work on offer 
2. Specific skills, knowledge and experience required 


3. Any special conditions; development possibilities 


Person specification 


1. Capacity to undertake a specific type of work or work role** 
2. Requirements for evidence of specific skills, knowledge and/or experience* * 
3. Capacity to work in certain type(s) of social situation or work environment and deal with certain 


type(s) of contingency 


* These may be expressed as types of qualification or specific qualifications. 
** These may be related to a specific type or level of qualification. 


Source: Authors. 


In WRL terms, where the intention is to bring 
together academic, vocational and work-related 
outcomes and requirements, the scope of these 
three perspectives can be defined as follows: 


e the nature and scope of the academic, social 
or occupational activities that an individual can/ 
is required to carry out, take responsibility for, 
collaborate on, and/or monitor and improve 
performance in — in the WRL system, these are 
referred to as ‘accountabilities’; 

e the type and complexity of intellectual and 
practical capabilities (deployable skills and 
knowledge) that an individual has/needs in order 
to carry out activities successfully — in the WRL 
system, these are referred to as ‘capacities’ and 


they include communication skills and the ability 
to access and use data as discrete elements; 

e the type of academic or occupational context 
(e.g. structured, stable, complex, strategic) 
in which individuals can/will have to carry out 
activities and in which they can identify and/ 
or investigate and/or deal with technical, social, 
ethical or environmental problems and issues 
that arise — in the WRL system, these relate 
to the capacity/need to deal with changes and 
challenges in the course of work or study and 
are referred to as ‘contingencies’. 


The use of these perspectives as an organising 
structure for the development and use of the WRLs 
is illustrated in Figure 1.2. Annex 1 provides details 
regarding each factor. 
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Figure 1-2. Organising structure of the WRLs 


Capacities 


Source: Authors. 


Clearly there are some overlaps between these 
perspectives. For example, the complexity of the 
contingencies that individuals have to deal with 
and the sophistication of the capacities they have 
to deploy will be related to the complexity of the 
responsibilities for which they are accountable. 
However, this is more likely to be an advantage 
than a disadvantage in creating a tool that can be 
used by individuals and organisations with different 
backgrounds, understandings and expectations. 

In all parts of the WRLs, the intention is to 
accommodate these differences while translating 
them into a lingua franca, and the overlaps — 
perhaps better seen as areas of tolerance — should 
allow users to capture aspects of capability in 
different ways and in different terms, and in this way 
assist in achieving the aim of the WRLs. 


The 11 WRL elements that were developed in this 
way are listed below (with short titles in bold). 


Accountabilities: carrying out and managing 
activities 


e scope and nature of activities 

e scope and nature of responsibilities 

e role in working with others 

e role in monitoring performance and learning to 
improve quality 


Accountabilities 


Contingencies 


Capacities: using skills, knowledge and know-how 


e scope and nature of skills and procedures 

e scope and nature of skills for communication 

e scope and nature of skills for accessing and 
using data 

e scope and nature of knowledge and know-how 


Contingencies: responding to recognised and 
emerging contextual factors 


e the nature of contexts of activity 

e role in researching and/or addressing problems 
and issues 

e role in addressing issues relating to values 


1.4 The WRL digital tool app 


The WRL digital tool (the VWWRL app), which is under 
development at the time of writing, is an instrument 
for translating qualifications, credentials or other 
sets of outcomes into WRL terms using the WRL 
element outcome statements. It will be a free- 
standing programme that can be downloaded for 
use on local computers or intranets. It will support 
users to produce a graphical WRL profile and (in the 
case of qualifications and credentials) a structured 
WRL report describing the qualification, credential 
or other outcome in terms that will be of use to 
other WRL users. 


Volume |: Thematic chapters 


19 


20 


Figure 1-3. WRL welcome page 


Name 


Organisation 


Position/role 


Email 


Source: Authors. 


The WRL digital tool will contain all the information 
required to create the graphical profile and will allow 
users to call up the terms and meanings in the WRL 
directory as they proceed. Throughout the process, 
users will be able to control both the sequence and 
the outcomes of the analysis to ensure that their 
qualifications, credentials or other sets of outcomes 
are accurately described and properly represented. 


The WRL profile will show where and how a 
qualification, credential or other outcome matches 
the relevant WRL elements and WRL stages. An 
illustrative example of a WRL profile is shown in 
Figure 1.4. 


To create a graphical profile, the Digital Tool will 
support users to decide, element by element, which 
stage is the best match for the outcomes they 

are translating into WRL terms and to record the 
evidence that backs up their judgements. It does 
this by posing a series of questions relating to the 
element and inviting users to select from a fixed 
menu of answers. In each case the answer may 

be ‘not fully relevant’. This will mean that any part 
played by the element in the qualification, credential 
or other set of outcomes is not significant enough 
to create an entry in the profile. The programme 
offers explanations from the WRL directory to 
explain the significance of many of the potential 


Welcome to the World Reference Levels (WRL) App 
The App will help you to translate any outcomes-based qualification, credential, entry requirements, job 


specification or framework level into an internationally recognised form of description which you can use 
in deciding on or negotiating recognition or progression arrangements. 


answers, but there is no restriction on the number 
or combination of answers that can be selected. 
The programme then uses a specially designed 
scoring system to suggest the appropriate stage 

for the element and proposes this to the user. The 
user can examine the relevant outcome statements 
and accept the proposal or reach a different 
conclusion. In either case, the user is invited to 
record evidence from the qualification, credential 

or other set of outcomes to back up the choice of 
stage. In the case of qualifications and credentials, 
the most important evidence will take the form of 
information about assessed outcomes. However, 
the evidence could also relate to the purpose, target 
holders/users, assumed capacities on entry, or other 
information about progression. 


The WRL digital tool will also gather information 
from the user to create a detailed structured report 
to back up and elaborate on WRL profiles for 
qualifications and credentials. In the WRL report, 
the WRL digital tool will organise information about 
the nature of the qualification, credential or other 
set of outcomes in a standard format. This report is 
intended to give enough information to allow those 
responsible to decide whether a qualification or 
credential meets their needs sufficiently to justify 
further investigation or other action. It will include 
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Figure 1-4. WRL profile 


Technical skills credential (awarded by Xxxxx) 


Stage element Al A2 


1. Activities 

2. Responsibilities 

3. Working with others 

4. Quality 

5. Skills & procedures 

6. Communication 

7. Data 

8. Knowledge & know-how 


9. Context 


10. Problems 


11. Values 


Source: Authors. 


a combination of fixed and free-form responses to 
questions about: 


e user status and contact information; 

e title, sector, purpose and intended learners of 
the qualification or credential; 

e the awarding, accrediting and/or standard-setting 
bodies, as appropriate; 

e the type, level, structure and size of the 
qualification or credential; 

e how it is intended to be delivered and assessed; 

e standards incorporated in or met by the 
qualification or credential; 

e information about assessment and quality 
assurance. 


The paperbased version of the WRLs will be 
accompanied by guidance on how information in 
qualifications or other specifications of capabilities 
should be matched against the components of the 
WRLs, and how the results of this matching should 
be reported to make them easy to understand and 
to use as a basis for discussions and negotiations. 
This ‘analogue’ approach might be supplemented by 
digital tools to support the matching process. 


B1 B2 C1 C2 D1 D2 


not fully relevant 


not fully relevant 


not fully relevant 


1.5 Conclusion and next steps 


The WRLs have been developed using international 
experience in defining and levelling learning 
outcomes. The WRLs do not replace national or 
regional qualifications frameworks and systems of 
quality assurance. They are a reference for better, 
fairer and more transparent recognition of skills and 
qualifications within and across borders. 


Largerscale piloting of the WRL digital tool is 
planned for 2019. This will be more extensive and 
inclusive than previous field-testing. The aim will be 
to evaluate both the functioning and the outputs of 
the WRL digital tool, and the pilots should lead to 
two types of report: (i) on the utility of the tool, and 
(ii) on the utility of the WRL profiles and reports. 
The pilots will cover as many sectors and types of 
qualification, credential and other set of outcomes 
as can be arranged. A number of types of pilot will 
be considered, including with bodies interested in: 


e using the WRL app to create and comment on 
WRL profiles and reports; 

e making national or international comparisons 
between qualifications or credentials with a view 
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to evaluating the role of the WRLs in establishing 
recognitions or pathways; 
e profiling framework level descriptors and 
commenting on the use of the WRL digital tool 
in comparing framework levels; 
e profiling programme entry requirements or job/ 
person specifications and comparing them with 
qualification and/or credential profiles. 


Annex 1. The three groupings of the 
WRL elements 


Accountabilities 

Scope and nature of activities: activities 

In the WRLs, an activity is any kind of academic, 
social or occupational task, procedure or process, 
or responsibility. An activity could be directed 

or allocated by another person, carried out as 

a normal function, or taken on voluntarily. The 
activity may only need to be carried out by the 
individual or it may involve planning, organising, 
supervising, managing or overseeing the work of 
others. Activities in the WRLs may be academic 
(e.g. acquiring knowledge, undertaking an enquiry 
to gather data, or communicating the results of 
academic research); social (e.g. contributing to 
the achievement of an objective, participating in 

a network, or motivating others to contribute to a 
campaign); or occupational, including training others. 


In broad terms, this element progresses: 


e from simple, highly structured activities that 
do not require knowledge or know-how that is 
specific to the field of the activity, 

e through complex technical activities that require 
different kinds of expertise, 

e to activities that are highly specialised, strategic 
or critical in their impact and require the creative 
use of advanced knowledge and know-how. 


Scope and nature of responsibilities: 
responsibilities 

In the WRLs, responsibilities relate to the nature of 
the activities for which the individual is answerable, 
the degree to which he/she is free to/required 

to make decisions about these, and the extent 

to which he/she will be expected to take or give 
guidance or instruction. 


In broad terms, this element progresses: 


e from carrying out activities under instruction 
(very limited responsibilities), 

e through taking responsibility for planning one’s 
own activities and mentoring, giving guidance 
on, supervising or coordinating the activities of 
others, 


e to taking full responsibility for planning, 
carrying out, evaluating and bringing about 
improvement in strategic activities across fields 
or organisations. 


Role in working with others: working with others 
This element is about collaboration and coordination. 
It deals with roles in, or relating to, formal or 
informal teams, groups and organisations (including 
academic communities, social networks, and 
divisions of large organisations). The collaboration 
may be practical or intellectual and could draw on 
expertise in the form of skills, knowledge and know- 
how, or insights. 


In broad terms, this element progresses: 


e from working with others under instruction, 

e through leading teams and groups, 

e to being a leader in the sense of shaping thought 
in a discipline, endeavour or occupation and 
collaborating with fellow experts and fellow 
leaders on strategic and critical activities. 


Role in monitoring performance and improving 
quality: quality 

This element covers the ability to monitor and 
measure the quality of performance against agreed 
or common measures and procedures and to take 
steps, though learning of different kinds, to maintain 
or raise the standard of activities or outcomes. The 
individuals concerned could be students, volunteers 
or employed persons and the performance could 
relate to either generic or specialised activities at 
any level of responsibility. The element assumes 
that every individual undertaking an activity will 
know, or have been directed towards, what 
constitutes quality in that activity and its outcomes 
and will have an interest in maintaining or improving 
the quality through academic, personal and/ 

or professional development. For convenience, 

this element is divided into two layers, one on 
monitoring and checking, reviewing or evaluating 
activities and their outcomes and the other on 
improving performance and outcomes through 
learning. ‘Learning’ here denotes any kind of 
intentional learning. It includes formal, non-formal 
and planned experiential learning and it may be 
carried out on instruction or independently. It 

may relate to activities, responsibilities, skills, 
procedures, knowledge or know-how and it may be 
designed to maintain, improve or extend existing 
academic, social or work-related capacities. 


In broad terms, this element progresses: 


e from a situation in which the individual is 
expected to check on their own performance 
using standards set for them, and to follow 
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further instructions to maintain or improve 
quality, 

e through the extension of responsibility to 
identifying the appropriate standards for 
evaluating their own performance and the 
performance of others, and to identifying and 
pursuing means of maintaining or improving the 
quality of performance, 

e to the situation in which the individual is acting 
as a reflective practitioner in his/her own 
activities, and has responsibilities for improving 
quality that are tied to some form of research, 
are more strategic, and apply across groups or 
organisations. 


Capacities 


Scope and nature of skills and procedures: skills 
and procedures 

In the WRLs, a skill is the ability to complete an 
activity satisfactorily. Skills may be academic, social 
or occupational. They may be intellectual, emotional 
or practical in nature. A procedure is a way of doing 
a specific activity — it might be a methodology, a 
technique or a practice. 


In broad terms, this element progresses: 


e from an emphasis on practical/technical skills 
and procedures, 

e toan emphasis on cognitive and creative skills 
and procedures, 

e from simple and routine skills and procedures, 

e to complex and highly specialised skills and 
procedures, 

e from the individual using a limited range of skills 
and procedures, 

e to the individual selecting from a broad range. 


Scope and nature of communication skills: 
communication 

This element covers the ability to use 
communication skills and procedures both to 
acquire information and ideas and to convey 
information and ideas to others. 


In broad terms, this element progresses: 


e from using basic or standard skills and 
procedures to access and record simple practical 
information and report it to a limited audience of 
colleagues and/or customers, 

e through selecting skills and modes of 
communication to gather, interpret and structure 
information and ideas and disseminate them to 
varied audiences, 

e to critically analysing and evaluating the 
significance of advanced ideas and presenting 
them in an appropriate form to diverse 


audiences with different interests in, and levels 
of understanding of, the topics and issues 
concerned. 


Scope and nature of skills for accessing and using 
data: data 

This element is about accessing, processing and 
evaluating numerical or other coded information. 

It includes raw and processed data, field data and 
experimental data. It is intended to reflect the use 
of data in roles that do not normally require the 
particular expertise of academics or professionals in 
mathematics, statistics or computing. 


In broad terms, this element progresses: 


e from using simple numeracy skills, procedures 
and programmes to access or record raw data, 

e through using a range of arithmetical and 
mathematical procedures and programmes to 
gather and process standard technical data for 
routine purposes, 

e to selecting or specifying advanced processes 
and programmes to generate and evaluate 
complex, technical and specialised data. 


Scope and nature of knowledge and know-how: 
knowledge and know-how 

In the WRLs, the terms ‘knowledge’ and ‘know- 
how’ are usually used together. ‘Knowledge’ is 
used to denote information and ideas that an 
individual can draw on or build on. ‘Know-how’ 
denotes explicit or tacit procedural knowledge and 
understanding. Both knowledge and know-how may 
be practical or conceptual, academic, social or work 
related, and they may be acquired formally, non- 
formally or informally. 


In broad terms, this element progresses: 


e from general knowledge (i.e. not related to the 
specific field of activity), through knowledge 
of a field, to knowledge that extends across a 
number of fields, normally at different levels, 

e from narrow to extensive knowledge and know- 
how, 

e from basic knowledge and simple know-how, 
through theoretical knowledge and conceptually 
based know-how, to the most advanced 
theoretical knowledge and theory-based know- 
how, 

e from using knowledge and know-how, to 
enhancing it and creating new insights. 


Contingencies 
The nature of contexts of activity: context 


In the WRLs, statements of context describe the 
conditions under which activities will be carried 


Volume |: Thematic chapters 


23 


out in terms that can apply to academic, social or 
occupational situations. The context may support, 
influence or restrict the activity. 


In broad terms, this element progresses: 


e from stable and highly structured contexts, 

e through contexts that are subject to change with 
different degrees of predictability, 

e to unsettled and problematic contexts which 
will require increasing degrees of adaptability, 
innovation, initiative and creativity. 


Role in addressing problems and issues: problems 
and issues 

In the WRLs, a problem is a difficulty or complication 
in an academic, social or occupational activity that 
requires some choice, adjustment or adaptation 

to allow the activity to proceed or be completed. 

A problem may be simple or complex, concrete or 
abstract, but in the WRLs it implies a direct difficulty 
of a kind that can reasonably be expected to be at 
least partly addressed immediately, adjusted for or 
resolved. At the lowest level, problems are likely to 
be routinely recognised in the area of activity, but at 
the next levels, problems may need to be scoped or 
defined before they can be addressed. In the WRLs, 
an issue is a deeper or more long-term complication 
in an academic, social or occupational field that is 
likely to require investigation to identify, clarify or 
define, and may take considerable time to carry out. 
Steps to address issues, which are likely to require 
the pursuit of quantitative or qualitative research, 
will normally be designed to produce reliable and 
durable results. 


Both problem-solving and research are seen in the 
WRLs to have in common systematic investigation 
to establish and evaluate data, reach conclusions, 
and take or recommend action. Both involve 
identifying, understanding, addressing and resolving 
complications that arise in undertaking activities at 
a particular level in academic, social or occupational 
contexts. 


In broad terms, this element progresses: 


e from dealing with familiar problems, or types of 
problem, that arise in standard activities, 

e through selecting and applying or adapting 
procedures for problem-solving and research, 

e through addressing increasingly complex and 
contingent problems and issues, 

e to developing new methods to deal with the 
most challenging and abstract issues. 


Role in addressing issues relating to values: values 
This element is about how and how far individuals 
can be expected to respond to ethical, social or 
environmental dimensions of their activities. It 
includes situations that may or may not be covered 
by relevant rules of practice or codes of conduct. 
These could arise in relation to academic, social or 
work activities in any field or at any WRL stage of 
progression. 


In broad terms, this element progresses: 


e from situations in which the individual is 
not expected to encounter ethical, social or 
environmental issues beyond those covered by 
existing policies and procedures, 

e to situations in which the individual will deal 
with ethical, social or environmental issues by 
using structured responses or reporting any that 
cannot be dealt with in this way, 

e through situations in which the individual may 
have to select the most appropriate response 
to problems, questions or issues that arise, 
drawing on existing codes of practice, 

e to situations in which the individual will have 
responsibility for addressing emerging issues 
that are not covered by existing policies, 
procedures or codes. 
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CHAPTER 2. 


BEYOND QUALIFICATIONS AS WE KNOW 
THEM TODAY: DIGITAL CREDENTIALS AND 


INTEROPERABILITY 


James Keevwy, JET Education Services, and Borhene Chakroun, UNESCO - Modified extract 
from Digital Credentialing: Implications for the recognition of learning across borders® 


Summary 


Digital technologies are creating new opportunities 
and challenges for skills development and 
recognition globally. Changes in modalities of 
access and learning methods, massification 

and internationalisation are taking place at an 
increasingly rapid pace. In this context, digital 
technology is also expected to offer new 
credentialing methods and systems that can 
capture, recognise and validate a broader range of 
learning outcomes in the era of lifelong learning. 
This paper will map the changing landscape of 
credentialing, identify key developments and chart 
the possible structure of a conducive and trustable 
ecosystem. The paper builds on a recent UNESCO 
publication in this field. 


2.1 Introduction 


Digital technologies are creating new opportunities 
and challenges for skills development and 
recognition globally. Changes in modalities of 
access and learning methods, massification 

and internationalisation are taking place at an 
increasingly rapid pace. In this context, significant 
attention is being given to the impact of technology 
on jobs and the demand for skills, and the risks of 
competition between robots and humans. Far less 
has been said about the opportunities that advances 
in digital technology will create for transforming 
education and training systems, including building 
new credentialing methods and systems that can 
capture, recognise and validate a broader range of 
learning outcomes in the era of lifelong learning. 


In today’s increasingly digitised economies and 
societies, accessing and understanding data about 


learning outcomes, skills and credentials is critical 
to achieving the 2030 Sustainable Agenda, including 
United Nations Sustainable Development Goal 4, 
with its particular focus on quality and inclusive 
education, recognising and offering lifelong learning 
opportunities for all. In this context, different 
stakeholders have different needs: education and 
training providers need learning data to build new 
programmes and learning pathways; employers 
need data to understand where to find qualified 
workers; and learners and workers need data to 
discern which learning pathways are more likely to 
lead to career opportunities. 


To date, there has not been an efficient national or 
global system for collecting, connecting, searching 
and comparing up-to-date information about learning 
outcomes and credentials in a common language 
or format that can be universally understood 

and easily accessed. This lack of information and 
systems contributes to confusion, lack of trust 
and uninformed decision-making regarding the 
recognition of skills and qualifications within and 
across borders. It also leads to talent loss for 
economies and employers. 


2.2 Digital credentials: Changing 
landscape 


Traditional degrees, or macro-credentials, have 
served an important purpose over many years to 
signal to employers that a graduate is employable. 
There is increasing dissatisfaction with qualifications 
(and the accompanying academic records) as a 
proxy for employability (Oliver, 2016). Some consider 
that ‘the transcript is dead’. Alternatives are being 
mooted, and in some instances have already 


6 Published by UNESCO in 2018. The original report was published as Open Access under the Attribution-ShareAlike 3.0 |GO 
(CC-BY-SA 3.0 IGO) licence (http://creativecommons.org/licenses/by-sa/3.0/igo/). By using the content of this publication, 
the users agree to be bound by the terms of use of the UNESCO Open Access Repository (www.unesco.org/open-access/ 


terms-use-ccbysa-en). 


been implemented, such as digital passports’, 
e-qualification (Chen-Wilson and Argles, 2010) 

and ‘3D CVs' (Oliver, 2016). The situation is also 
impacted by the growing digitisation of credentials, 
also referred to as the advent of ‘micro-credentials’ 
(Ifenthaler et al., 2016). Recent developments and 
initiatives, including the Groningen Declaration 
Network, the work of the Post-Secondary 
Electronic Standards Council, the Common Student 
System in Norway, and initiatives in other Nordic 
countries, point to the importance of mapping the 
landscape, identifying key actors and charting future 
developments with partners. 


At the core of this process is the growing move 
towards digital credentialing, which has critical 
implications for the recognition of learning within 
and across borders. This move is directly attributed 
to the length, cost and perceived low return on 
investment of traditional degrees (referred to 

as macro-degrees), even when offered digitally, 
whereas smaller chunks of learning (referred to 

as micro-credentials) provide greater flexibility, 
especially when offered digitally. 


There is a wide range of definitions and 
understandings of digital credentials. The European 
Commission speaks about ‘digitally signed 
credentials’, or badges. These visual tokens of 
achievement are described as a ‘new way to capture 
and communicate what an individual knows and 
can demonstrate’ (Finkelstein et al., 2013, p. 1, in 
Mah, 2016), effectively providing a technological 
solution to the problem of representing learning 
beyond qualifications. Badges can be collected 
through social media and other platforms, such as 
LinkedIn, Jive, Fidelis, Credly and Mozilla, but also 
through more formal associations with established 
institutions, such as Coursera, affiliated with 
Stanford University, Open2Study, affiliated with 
Open Universities Australia, and Passport, affiliated 
with Purdue University (Oliver, 2016, in Mah, 2016). 
Multinationals, including IBM and Accenture, are 
also increasingly developing their own badges for 
both their staff and the wider public. 


Badges are viewed as examples of micro- 
credentials, representing discrete skill sets that 

can be grouped or ‘stacked’ to form a larger, or 
macro-, credential. Oliver (2016) suggests that digital 
credentials that use badging have the following 
benefits. 


7 — https://europass.cedefop.europa.eu/, last accessed 8 June 
2017. 


e Granular: More than simply communicating 
marks and grades, they can pinpoint where 
skills and competences — for example, 
innovative thinking and teamwork — have been 
demonstrated. 

e Stackable: Because they are digital, they can 
be added to credential repositories, mapped 
to qualifications frameworks, and more easily 
understood in terms of credit eligibility towards 
other credential systems. 

e Evidentiary: They can point the reader of the 
credential directly to learning evidence created 
by the learner. 

e Personalised: They can more accurately 
represent each learner's achievements, 
highlighting where skills or outcomes were 
achieved above the minimum standard. 

e Machine-readable: If built using open technical 
standards, they enable rich analytics, showing, 
for example, which graduates in a cohort 
excelled in communication skills or teamwork. 


While there is a strong push towards investing, 
developing and using digital badges on a large scale 
by businesses, universities and training providers, 
several issues and limitations remain. 


e Security: As in other sectors, it is still easy to 
forge many things online, from the identity of 
the learner to the veracity of test responses and 
the bona fides of the granter of the badge or 
certificate. 

e Users’ perception: A recent survey by Extreme 
Networks showed that 46% of participants 
believed that digital badging is not yet widely 
recognised, and 34% of participants did not 
fully understand the concept. The survey also 
showed that over 60% of participants believed 
that digital badges would eventually either 
supplement or entirely replace diplomas and 
course certificates®. 

e Quality assurance and transparency: The 
mass awarding of badges with little or no quality 
assurance and the wide range of sources of 
badges will affect their credibility and visibility, 
with the risk of what can be called a ‘jungle of 
badges’. 


e Access to the internet: |ndividuals living in 


developing countries face additional problems 
relating to internet access, including lack of 
infrastructure; low incomes and affordability; 
limited user capabilities, including basic literacy 


8 See https://content.extremenetworks.com/ex- 
treme-networks-blog/how-we-use-digital-badg- 
es-forrecognition-and-motivation-at-extreme-net- 
works?_ga=2.182621414.1133571021.1517737521- 
894459921.1517737521 (last accessed 15 April 2019). 
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and digital literacy; and reduced incentives for 
access, including lack of awareness, of relevant 
content, and of cultural or social acceptance 
(Schmida et al., 2017). 


These aspects are echoed by the EU digitally signed 
credentials framework, which includes the following 
guiding principles: 


user centricity; 

. inclusion and accessibility; 

. openness; 

data protection; 

. interoperability; 

transparency; 

resilience; 

. reusability; 

. qualifications as a public good. 


OMNATRWNS 


2.3 The possible structure of the 
ecosystem 


In a recent publication (Keevy and Chakroun, 2015), 

UNESCO suggested an ecosystem model for digital 
credentialing. This is made up of seven interrelated 

sectors and groups of stakeholders, anchored 


Figure 2-1. The digital credentials ecosystem 


to specific functions in the digital credentials 
environment. 


The elements of this model are as follows: 


Use. These are the users of credentials, notably 
learners, who are placed at the centre of the 
system (AACRAO, 2014). Providers can also be 
users, aS can employers. 

Provide. This refers mainly to education and 
training institutions and the emerging variety of 
forprofit and non-profit digital platforms, such as 
Coursera, FutureLearn, Credly, Verifdiploma and 
Mozilla. 

Award. Awarding bodies in the traditional sense 
are institutions and professional bodies. To this 
list we need to add employers, massive open 
online courses (MOOCs) and, in some instances, 
the owners/hosts of digital platforms such as 
IMS Global. 

Quality assure. This is where the line between 
macro- and micro-credentials is probably the 
clearest. The lack of quality assurance poses 

a significant threat to the credibility of digital 
credentials, and also sets constraints on the 


Standards 


Source: Keevy and Chakroun (2015). 


Quality assure 
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flexibility of traditional degrees®. The issues 
of trust, and particularly authentication and 
authorisation, are critical in this context. 

e Evaluate. The evaluation of credentials has 
been owned by credential evaluation agencies, 
such as the ENIC-NARIC (ENIC is the European 
Network of Information Centres in the European 
Region, while NARIC is the National Academic 
Recognition Information Centres in the European 
Union) network and some qualifications 
authorities. The value judgements required 
(Bai-Yun, 2017) and the relative opaqueness 
of the methodologies employed have resulted 
in some level of protection, but this has been 
challenged by the increasing use of learning 
outcomes in qualifications frameworks nationally 
and regionally, and, at present, also by the new 
forms of credential that are the topic of this 
paper. 

e¢ Verify. The range of both public and private 
verification agencies that have emerged in the 
past five years has increased substantially, 
and can be directly attributed to the benefits 
associated with the digitisation of credentials. 
The notion of a clearinghouse is closely 
associated with this function (Torres, 2017). 

e Convene. The last sector is also the most 
critical. International agencies such as UNESCO 
and the International Labour Organisation (ILO) 
have a role to play and, increasingly, so too do 
open communities and networks that have 
developed organically and comprise an eclectic 
mix of actors. 


The ecosystem also hinges on the range of tools 
and technologies available. Dowling’s (2018) analysis 
of the different architectures available, covering 
their scope and functionality, impact on mobility 

of learners, security and participation, is highly 
relevant in this context. Table 2.1 presents the key 
advantages and challenges. 


8 In 2016 the Open Badge for Education Extensions (OBEE) 
Initiative’s work centred around exploring how the addition 
of ‘Issuer Accreditation’ and ‘Assessment’ extensions to 
the Open Badge specification might help communicate 
the rigour with which badge earners’ activities were scru- 
tinised before a decision was made by the badge issuer 
to award the badge. The Issuer Accreditation extension 
will provide a reference to single or multiple accreditation 
bodies that certify the badge issuer. The Assessment 
extension will provide information about single or multiple 
assessments that are required as part of the badge issu- 
ance process. 


2.4 Recent international developments 


In Europe, in addition to the seminal work on the 
European Qualifications Framework, the European 
Commission adopted the Digital Education Action 
Plan in 2018 (European Commission, 2018) with 

the goal of supporting technology use and digital 
competence development in education and 
launching the work on digitally signed qualifications. 
The Action Plan suggests the need to provide a 
framework for issuing digitally certified qualifications 
and validating digitally acquired skills that are 
trusted and multilingual and that can be stored in 
professional profiles (CVs) such as Europass. 


In the USA, the Lumina Foundation launched a 
centralised credential data platform called the 
Registry in December 2017 (Lumina Foundation, 
2016; Lumina, 2017). This includes a common 
credentialing language for credential evaluation, 
a digital application to search for credentialing 
information, and an application-programming 
interface (API) tool to allow organisations to 
continuously upload up-to-date information 

to the registry. The Credential Engine aims to 
gather credentialing information from all types of 
source, including degrees, certificates, badges, 
apprenticeships, licences, micro-credentials 
and PhDs, thus drastically improving credential 
transparency. 


The Credential Engine includes the following 
features: 


¢ Common language: New metadata called the 
Credential Transparency Description Language 
(CDTL) will be used to describe key features of 
credentials. 

e Open-licensed registry: This voluntary registry, 
the first of its kind, will share comparable 
information from credentialing organisations 
about their range of credentials and how they 
relate to each other, to help individuals create 
learning pathways. 

e Shareable data: Customised apps can be built 
for students, companies and other interested 
parties, making the massive database even more 
useful. 


In China, the China Higher Education Student 


Information and Career Centre has pioneered 
student data digitisation in the country. Its database, 
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which includes students from across the nation, 
contains 864 million pieces of data, and each year 
100 million more pieces of information are added. 
Its qualification-verification service has checked 
70 million student records, and produced 4 million 
online and 1 million paper verification reports 
(AACRAO, 2014, p. 12). 


2.5 Privacy challenges 


The EU General Data Protection Regulation (GDPR) 
is one of the most ambitious, bold and impactful 
developments in privacy matters. It is bound to 
change the face of many practices worldwide 
because of its extraterritorial scope and the scale of 
the fines that data protection authorities can enforce 
in case of non-compliance (2-4% of companies’ 
worldwide turnover). 


The regulation applies to any organisation worldwide 
that is processing the personal information of EU 
residents (whereas the previous Data Protection 
Directive only applied if the organisation had 
infrastructure within the EU). Taking the example 

of educational institutions, even those without a 
physical presence in Europe are likely to interact 
with students based in the EU if they run distance- 
learning programmes. The same applies to private 
online education platforms. 


In the USA, despite a patchwork of already existing 
privacy-protection laws, some states have recently 
passed laws regulating ed-tech companies, 
specifically protecting students’ online personal 
information. 


The first initiative of this kind was the California 
Student Online Personal Information Protection 
Act (SOPIPA) (Centre for Digital Education, 2019), 
which took effect in 2016. The scope of the act 

is limited to websites, services or applications 
‘used primarily for K-12 school purposes and [...] 
designed and marketed for K-12 school purposes’. 
The scope is thus limited, as It is explicitly not 
applicable ‘to general audience Internet Web sites, 
general audience online services, general audience 
online applications, or general audience mobile 
applications’. Personal information goes beyond 
classic attributes, to include ‘text messages, 

[...] search activity, photos, voice recordings, or 
geolocation information’. Responding to the fear that 
learners’ educational data would be used for profit, 
the law forbids the operator of the ed-tech service 
to ‘engage in targeted advertising’ on its website, 
service or application, as well as ‘on any other site, 
service, or application when the targeting of the 
advertising is based upon any information’ acquired 
through the learner’s use of the operator's service. 


The law also prohibits the selling of students’ 
information. 


2.6 Looking to the future 


It is abundantly clear that any form of digital 
credential ecosystem comprises a combination 

of more traditional and bette-developed systems 
and more disruptive and, for the most part, less- 
developed systems. This interplay allows for 
innovation, but also creates a vacuum in which the 
learner/user is vulnerable. As countries, regional 
economic communities and the international 
community struggle to develop a unified strategy 
to ensure better and fair recognition of skills and 
certification across borders, taking advantage of 
technological progress and innovations linked to the 
recognition of learning and learners’ records cannot 
progress without commonly agreed digital metadata 
standards for such records. 


Policy-makers will need to take seriously this 
international trend towards digital credentialing, 
digital standards and, increasingly, interoperability, 
or risk being overtaken by new and open learning 
recognition systems in the near future. 
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CHAPTER 3. 


THE ROLE OF LEARNING OUTCOMES 
IN GOVERNING AND REFORMING EDUCATION 
AND TRAINING: REFLECTIONS ON STRENGTHS 


AND LIMITATIONS 


Jens Bjorndavold, Cedefop” 


Summary 


The purpose of this article is to reflect on the role 
played by learning outcomes in governing and 
reforming education and training. Following an 
initial discussion of the aspirations underpinning 
the learning outcomes approach, the article 
reflects on the conceptual assumptions informing 
initiatives in this area. This provides the basis for a 
final discussion of the strengths and limitations of 
learning-outcomes-based governance at different 
levels (overall policy coordination, institutional 
management and pedagogical reform). 


3.1 Introduction 


The learning outcomes principle, emphasising 

what a learner is expected to know, be able to do 
and understand, now underpins most European 
education and training systems (Cedefop, 2016). 

A recent survey" of 1 500 vocational education 

and training (VET) stakeholders and experts in 30 
European countries shows that the shift to learning 
outcomes is considered one of the most significant 
trends influencing European VET over the past two 
decades. Similar developments can be observed in 
higher education (Michelsen et al., 2016), where the 
Bologna process, for example, in relation to credit 
transfer and quality assurance, has promoted a shift 
from an input- to an outcome-oriented approach. 
The adoption of the European Qualifications 
Framework (EQF) in 2008 played a particularly 
important role in this reorientation, triggering 

the establishment of comprehensive, learning- 
outcomes-based national qualifications frameworks 
(NOFs) in all European countries (Cedefop, 2018). 
These latter developments have allowed for 
coordinated national approaches, resulting in a more 
systematic implementation of learning outcomes. 


10 Thanks to Slava Pevec Grm and Anastasia Pouliou, both of 
Cedefop, for their valuable comments and suggestions. 

"See www.cedefop.europa.eu/en/events-and-projects/ 
projects/changing-nature-and-role-vocational-educa- 
tion-and-training-vet-europe (last accessed 15 April 2019). 


While questioned and contested by some (Hussey 
and Smith, 2003, 2008; Allais, 2014), the learning 
outcomes principle is now firmly embedded in 
European education and training policies and 
(possibly to a lesser degree) practices. 


Overall, strengthening the learning outcomes 
principle forms part of an effort to reform and 
modernise education and training systems, their 
institutions and their associated teaching and 
learning practices. With some overlap, governance 
inspired by learning outcomes can be observed 

in three distinct areas. First, the shift to learning 
outcomes is seen as a policy tool that makes it 
possible for national authorities to better shape 
the orientation and profile of their education and 
training systems. This is related partly to the 

need to increase the transparency of systems for 
users, and partly to the need to strengthen the 
accountability of providers. Second, the shift to 
learning outcomes is seen as a managerial tool 
that allows for the steering of complex education 
and training institutions. To some extent inspired 
by management by objectives (MBO) and 

new public management (NPM) traditions, the 
learning outcomes approach is seen as a way of 
strengthening the overall quality and coherence of 
the programmes delivered by institutions. In some 
universities, for example, we can observe that 
learning outcomes inform and influence institutional 
governance (Michelsen et al., 2017). Third, the shift 
to learning outcomes is seen as a pedagogical 

tool and as a way of promoting a learnercentred 
approach to education and training. By focusing on 
what a learner is expected to know, be able to do 
and understand, the learning outcomes approach 
is seen as challenging an understanding of learning 
as the transmission of knowledge from teachers to 
passive learners. 


3.2 Aspirations and ambitions 
Although pioneered by a few anglophone countries 


in the 1980s and 1990s, and mostly motivated by 
internal national challenges in relation to VET, the 


current prominence given to learning outcomes in 
the majority of European countries has developed 
over a period of less than two decades. While VET 
can be seen as a frontrunner, higher education and 
(to a lesser extent) general education institutions are 
rapidly catching up (Cedefop, 2009; 2016). European 
policy initiatives such as the EQF and the Bologna 
process have, as stated above, contributed to these 
developments, frequently playing a decisive role in 
articulating the ambitions behind the shift to learning 
outcomes. These aspirations and ambitions differ 
between contexts and levels but are, in most cases, 
interconnected and overlapping. 


3.2.1 Learning outcomes and national/European 
policy reform 

Learning outcomes underpin a number of policy 
initiatives at national and European level (Souto- 
Otero, 2012). The growing importance of the 
validation of non-formal and informal learning, for 
example, is fundamentally inspired by the learning 
outcomes approach. All learning outcomes, 
irrespective of when and where they have been 
acquired, should, in principle, be made visible 

and appropriately valued. Validation promises to 
reduce the barriers between learning in education 
and training and learning at work, and to enable 
more flexible learning progression throughout life. 
The rapid implementation of learning-outcomes- 
based qualifications frameworks in the past decade 
captures the high ambitions associated with the 
reorientation towards outcomes. Frameworks 

are expected to support learners by providing a 
more transparent overview of qualifications and 
to support providers by facilitating coordination 
and cooperation across institutional and national 
borders. Increasingly, we also see efforts to link 
qualifications frameworks and quality assurance 
mechanisms, where the learning outcome 
descriptors of the frameworks provide a reference 
point for the levelling and profiling of programmes 
and qualifications. In essence, the overall policy 
aspirations can be summarised as follows. 


e The shift to learning outcomes is the key to 
lifelong and life-wide learning. 

e The shift to learning outcomes facilitates the 
valuing of all learning, including that which has 
been acquired outside formal education and 
training. 

e The shift to learning outcomes increases the 
transparency of qualifications for learners and 
employers. 

e The shift to learning outcomes facilitates 
progress in an increasingly complex and diverse 
education and training landscape. 

e The shift to learning outcomes strengthens 
the accountability of education and training by 


focusing on actually achieved learning outcomes 
rather than input factors and intentions. 

e The shift to learning outcomes increases the 
comparability of qualifications between countries 
and facilitates the transfer and accumulation of 
learning outcomes across borders. 


3.2.2 Learning outcomes and institutional 
management 

The role of learning outcomes as a management 
tool that is relevant to the steering and coordination 
of complex organisations has come to the fore 

in higher education. Learning outcomes are used 

at policy level as a tool for governing, managing 
and reforming education and training. Some see 
learning outcomes as a way to manage regional 
and local practices centrally, strengthening the 
accountability of schools and teachers. Others 

see them as a way to put the focus on learners, 
providing teachers and students with tools for 
active and open learning. These policy choices, 
pointing in somewhat different directions, illustrate 
the relationship between the learning outcomes 
approach and management principles used 

in other policy areas (for example, MBO). The 
question arises whether and how these divergent 
approaches can be balanced. As indicated by 
Bleiklie et al. (2017), the shift to learning outcomes 
is seen as critical for transforming universities from 
loosely coupled to more managerially integrated 
institutions. While the learning outcomes approach 
has traditionally been linked to pedagogies and 
theories of learning, the general focus on outputs/ 
outcomes and results links it to important trends 

in management theory. MBO was established as 

a major approach to governance from the 1960s 
and onwards (Drucker, 1952) and is closely linked 
to NPM in political science and organisational 
theory. Viewed within this broader context, the 
implementation of learning outcomes is seen as a 
way of enabling local autonomy and responsibility 
within a centrally defined frame of broader strategic 
objectives. This effort to ‘square the circle’ is well 
illustrated in VET (Cedefop, 2018), where national 
qualification standards and programme descriptions 
explicitly leave room for local adaptation. In some 
higher education institutions, the learning outcomes 
approach has been used to promote a more 
collective management (in some cases, a stronger 
role for university management), replacing the 
traditional approach in which individual teachers and 
lecturers took full responsibility of their own courses 
(Bleiklie et al., 2017, p. 77). In VET in particular, the 
shift to learning outcomes is also seen as a way to 
better interact with stakeholders outside education 
and training, notably as a way to strengthen the 
relevance and quality of the programmes and 
qualifications on offer. In essence, the following key 
aspirations can be identified. 
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e The shift to learning outcomes is not an isolated 
phenomenon, but is closely linked to broader 
management traditions in the private as well as 
the public sector. 

e The outcome orientation shared with traditions 
such as MBO and NPM allows for a combination 
of strategic steering from the top and extensive 
opportunity for local autonomy and adaptation. 

e The shift to learning outcomes allows for 
the continuous updating and renewal of 
qualifications reflecting the needs of the 
labour market and society and strengthens the 
relevance and quality of qualifications. 


3.2.3 Learning outcomes and pedagogical reform 
Learning outcomes are frequently seen (Adam, 
2004) as a way to transform teaching and learning 
from an input-, teachercentred to an outcome-, 
learnercentred approach (Sin, 2014). Indeed, much 
of the conceptual work on learning outcomes has 
developed within a broader pedagogical tradition 
(Lassnigg, 2012) that seeks to improve the quality 
and relevance of teaching and learning processes. 
The role of learning outcomes in reforming 
pedagogics is centred on four key aspirations. First, 
learning outcomes increase transparency and make 
it possible for the learner to better understand the 
learning process and the expectations involved. An 
unambiguous writing of learning outcomes will, 

it is asserted, benefit teachers and trainers also. 
Second, the ‘alignment’ of teaching, learning and 
assessment is seen as critical if learning outcomes 
are to make any difference. As Sweetman (2017) 
points out, ‘constructive alignment’ represents an 
effort to make teaching and learning processes 
more explicit, facilitating a more active role for 
learners and students. Third, as aimed for in 
‘constructive alignment’, the shift to learning 
outcomes balances the relationship between 
teacher and learner and allows for a more active 
involvement of the latter. Focusing on the learner 

is seen by some as a condition for active learning, 
flexible learning, problem-based learning and self- 
directed learning. While strengthening the focus 

on the learner, this rebalancing also changes the 
perception of what teachers should do, emphasising 
their role as facilitators and enablers. In accordance 
with this perspective, learning should not be 
reduced to a process of transferring knowledge to a 
passive recipient. Finally, some of these approaches 
see the use of learning outcomes as a way of 
‘teacherproofing’ education. Clearly defined learning 
outcomes, it is asserted, would reduce the harm 
done by substandard teaching and create a level 
playing field. In essence, the pedagogical aspirations 
can be summarised as follows. 


e The learning outcomes approach clarifies the 
intention of the teaching and learning process. 


e The learning outcomes approach clarifies 
what Is expected from the learner in terms of 
knowledge, skills and overall competences. 

e The learning outcomes approach facilitates, 
through a consistent alignment of teaching, 

earning and assessment, a better design of 

education and training programmes. 

e The increased transparency and clarity of the 

earning outcomes approach facilitates a more 

earnercentred process, potentially opening 
up to active, self-directed and problem-based 
earning. 


3.2.4 Learning outcomes - the ‘Swiss Army knife’ 
of education and training reform? 

As demonstrated above, the shift to learning 
outcomes is seen as a core component of education 
and training reform, addressing policies at national 
and European levels as well as the interaction 
between teachers and learners at local level. 
Resembling a Swiss Army Knife, an implement 

that is suitable for a variety of purposes, learning 
outcomes have also been described as the glue that 
binds together diverse initiatives at different levels 
and in a wide range of areas. 


The impact of learning outcomes in policies and on 
learning is directly dependent on two key factors. 
First, the conceptual and theoretical understanding 
of learning outcomes influences the way they are 
interpreted and translated into practice. Critics 

have pointed out that some articulations of learning 
outcomes potentially lead to a ‘dumbing down’ 
(Allais, 2012) of the learning process rather than an 
opening up to active and self-directed learning in the 
way indicated above. Second, learning outcomes 
must be embedded in policies and practices if 

they are to make any difference. If the approach is 
regarded as external to the day-to-day challenges 
faced by policy-makers and teachers and trainers, it 
could easily be seen as a merely symbolic piece of 
hype that can be ignored. The following sections will 
address these issues in more detail. 


3.3 The conceptual basis of learning 
outcomes 


Not surprisingly, given the expected impact of 

and the high ambitions for learning outcomes, the 
conceptual basis of the approach is debated and, to 
some extent, contested. 


3.3.1 A behaviourist bias? 

The Cedefop (2017) Handbook on the definition, 
writing and application of learning outcomes points 
out that the learning outcomes approach is seen by 
some (e.g. Campbell, 2014) as implicitly favouring 
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a reductionist and behaviourist understanding of 
learning. According to this criticism, the learning 
outcomes approach risks reducing the richness of 
learning by imposing a simplistic stimulus-response 
paradigm of learning in which only observable 

and measurable outcomes count. This, according 
to critics, assumes a linear and overly simplistic 
learning process where complex activity verbs (such 
as ‘understand’) should be avoided and replaced 

by narrower terms with clear borderlines. Allais 
(2012, 2014) repeats this criticism with reference 

to the way in which knowledge is treated ‘as 
information that can be divided into little bits that 
can be selected and combined at will’ (Allais, 2014, 
p. 39)’. She believes that this ‘ignores the extent 
to which knowledge is organised in bodies of 
hierarchical conceptual relationships’ (Allais, 2014, 
p. 39), and that the value of such knowledge does 
not respect the conditions in which knowledge is 
acquired. These arguments reflect the criticism of 
‘educational technologies’ developed in the USA 

in the 1960s. The aim at that time was to create 
‘teacherproof’ education and training systems, 
ensuring predictability of delivery and thus equal 
chances for everyone. While objectives relating to 
predictability and equality are still of key importance, 
the belief that education and training systems can 
be fully controlled from the top has, indeed, been 
questioned. 


Many researchers (e.g. Dobbins, 2014) argue 
against the assumption that the shift to learning 
outcomes by default implies this form of 
reductionism. Learning outcomes can, on the 
contrary, focus on a wide range of knowledge, 
skills and competences; while some of these 

may be behavioural in character (using a particular 
tool for a particular purpose), others imply more 
complex and ambiguous processes (linked to the 
critical evaluation of arguments supporting a policy 
decision) (Dobbins, 2014, p. 2). Biggs (1999; Biggs 
and Tang, 2007) pursues this point and states that 
in the design of learning outcomes and assessment 
tasks, teachers should be free to use open-ended 
verbs such as ‘design’, ‘create’, ‘hypothesise’ 

and ‘reflect’, and that this is a way to avoid the 
predetermined or rigid design of teaching and 
assessment. A key element in this understanding is 
that learning outcomes need, to some extent, to be 
ambiguous and to leave room for local and individual 
interpretation and (not least) communication. This 
inherent ambiguity and openness is consequential 
to our understanding of the reforming potential of 
learning outcomes. 


2 See also O’Brien and Brancaleone (2011). 


3.3.2 Different rationalities 

Michelsen et al. (2016) point to two contrasting 
positions that are relevant for analysing the impact 
of learning outcomes as a policy tool. First, it is 
possible to apply an instrumental perspective, 

in which implementation and impact are viewed 

in terms of ends-means rationality. Also termed 
‘strategic-instrumental rationality’ (Eriksen, 1995), 
this perspective, to some extent, underpins the 
management traditions referred to above (MBO, 
NPM), where impact is measured in relation to 
the ability of actors to choose optimal measures 
to reach a fixed goal. This somehow implies that 
objectives are known; that implications of choices 
are known; that options are ranked; and that the 
option giving the best result is chosen. This model 
of rationality, inspired by micro-economics and, to 
some extent, behaviourist management traditions, 
tends to underestimate the complexity of the 
policies and practices addressed. Shifting to learning 
outcomes Is, for the reasons listed above, seen 
by some as a ‘managerial turn’ and as imposing 
governance principles that are unsuitable to the 
complex reality of education and training. 


Second, an institutional perspective offers a 
contrasting position in which instrument and context 
are considered to reciprocally relate. This points 

to the important role played by policy-makers, 
managers, teachers and learners in interpreting 
learning outcomes, emphasising the broad, open 
and somewhat ambiguous character of such 
outcomes. The institutional perspective, rooted in 
political sciences and organisational theories, can 

be further elaborated and deepened by applying a 
communicative and/or deliberative form of rationality 
in which communication and deliberation play a key 
role. This implies that rationality is not only about 
choosing the best means to reach a predefined aim, 
but also about being able and willing to present 

the reasons for one’s own preferences. Learning 
outcome statements should be seen not as 
absolute (objective) rules predetermining learning 
but as a reference point for dialogue, interpretation 
and, in some cases, innovation. 


This implies taking the ambiguous character of 
earning outcomes as a positive starting point. 
Without this clarification it would be easy to 
overstate the potential impact of learning outcomes, 
in particular by overestimating the extent to which 
top-down implementation of learning outcomes can 
(or even should) influence teachers, trainers and 
earners. 
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3.4 Impact and added value of learning 
outcomes: What evidence exists? 


As already indicated, the current prominence 

of learning outcomes is a recent phenomenon. 
Consequently, there is limited evidence on the 
actual impact of this perspective on overall policies, 
institutional management and pedagogics. Only 
in recent years have we seen the emergence 

of empirical research that tries to trace more 
systematically the strengths and limitations of 
the learning outcomes approach. Some of these 
findings are helpful in better understanding the 
realism of the aspirations and ambitions listed 
above. 


3.4.1 The impact of learning outcomes on policy 
reform 

Cedefop's two studies on the shift to learning 
outcomes (2009; 2016) are the only ones that 
provide a broad and comprehensive overview of 
the implementation of learning outcomes across 
the different subsectors of education and training 
(general, vocational and higher education and 
training). Using the results of the 2009 study as 

a baseline, the 2016 study shows that significant 
progress had been made in the seven years 
covered. The report states that ‘while a few years 
ago the shift was most visible in the VET and adult 
education subsystems, now it is highly visible 

in higher education in almost all countries and, 

in several, in primary and secondary education’ 
(Cedefop, 2016, pp. 21-22). These developments, 
it is pointed out, are closely related to the 
development and implementation of NOFs in 
almost all European countries. Notably triggered by 
the launch of the EOF in 2008, these frameworks 
provide a tool for more systematic implementation 
of learning outcomes across the subsystems of 
education and training. The processes involved in 
developing comprehensive frameworks provide 

an important reference for communication 

and interaction between diverse stakeholders. 
Qualifications frameworks, building on existing but 
fragmented national learning outcomes approaches, 
have focused minds and provided a platform for 
cooperation in this area. The impact of European 
initiatives in this area is particularly visible in higher 
education, where from 2005 onwards the Bologna 
process has already explicitly promoted the learning 
outcomes approach, linked not only to frameworks 
but also, significantly, to credit systems (European 
Credit Transfer and Accumulation System (ECTS)) 
and quality assurance (European Standards and 
Guidelines). 


While qualifications frameworks can be seen as 
important instruments for promoting learning 


outcomes, in effect changing the national 
institutional landscape, it can still be questioned 
whether these developments make a difference 

to education and training practices. More recent 
Cedefop studies point to two forms of ‘deeper’ 
impact. First, learning-outcomes-based frameworks 
have increased the visibility and transparency of 
education and training systems. The introduction 
of NOFs with explicit learning-outcomes-based 
levels have helped to make national education 

and qualification systems more readable and 
easier to understand within and across European 
countries (Cedefop, 2018). Introducing a common 
learning-outcomes-based language for describing 
qualifications across education and training 
subsystems, the national frameworks now provide 
a comprehensive map of national qualifications and 
relationships between them. Although varying in 
coverage and quality between countries, progress 
has been significant in recent years'°. 


Second, in some countries the introduction of 
learning-outcomes-based frameworks has triggered 
systemic reforms. This is exemplified by Estonia, 
where a lack of initial qualifications at NOF level 5 
was identified through the development of an 
overarching framework. The main discussion 
centred on the fact that there were no initial 
education and training qualifications at this level. 
The introduction of learning-outcomes-based 
frameworks has been particularly important in what 
can be termed ‘higher VET’. Pioneered by Germany, 
the learning-outcomes-based levels have been used 
to strengthen the visibility of this particular part 

of higher education, exemplified by the placing of 
the Master Craftsman qualification at level 6 of the 
EOF and the establishment of a new vocational 
(‘Beruf’) Master at EOF level 7 from 2019 onwards. 
Such developments, which can also be observed in 
countries such as Switzerland, Austria and Norway, 
signal a rebalancing of education and training 
systems through which vocational and professional 
aspects are given higher visibility and priority. In 
general, NOF level descriptors are increasingly 
used as ‘yardsticks’ to review the content and level 
of qualifications. Comprehensive and integrated 
qualification registers increasingly underpin the 
NOFs and make information on qualifications 
accessible to students, employers and guidance 
staff'*. The case of Portugal demonstrates how NOF 
level descriptors can be used to support the review 
and renewal of qualifications. 


13 See EOF referencing reports: https://ec.europa.eu/ploteus/ 
sl/documentation (last accessed 15 April 2019). 

4 See, for example, the German qualifications database: 
www.dar.de/content/2316.php; and the Slovenian qualifi- 
cation database: www.nok.si/en/ (last accessed 15 April 
2019). 
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The validation of non-formal and informal learning 
is, perhaps, the best example of how the learning 
outcomes perspective is influencing education and 
training in practice (Cedefop and EU Commission, 
2015). Validation, or the effort to make visible and 
to value the learning taking place outside formal 
education, is gradually becoming an integral part 
of national education and training and lifelong 
learning arrangements in Europe. The increasing 
use of learning outcomes and attempts to make 
qualification systems more coherent and more open 
to learning that occurs outside the formal contexts 
of schooling and other establishments constitute a 
step towards a life-wide learning approach. Almost 
all countries have now put in place arrangements 
that allow for the acquisition of full or partial 
qualifications through the validation of non-formal 
and informal learning. Several countries are also 
working towards comprehensive systems offering 
validation opportunities where people live, work 
and/or study. The learning outcomes approach is 
fundamental to these developments, emphasising 
that in principle, all learning, irrespective of its origin, 
needs to be taken into account. Furthermore, the 
approach puts pressure on countries to establish 
clear, learning-outcomes-based standards and 
reference points for validation. The support for 

this shift in perspective is not a given. In some 
countries, the quality of these approaches has 
been questioned, sometimes signalling distrust 

of learning outside formal education and training. 
Overall, validation can be seen as the application of 
learning outcomes with the most direct impact on 
the lives of individual citizens. 


Returning to the aspirations and ambitions listed 
above, some initial conclusions regarding impact 
can be made. The application and promotion of 
learning outcomes by comprehensive qualifications 
frameworks and validation arrangements illustrates 
how the aspirations of lifelong learning, the 

valuing of all learning and the transparency and 
comparability of qualifications are being addressed. 
While they are only indicative of potential longer 
term impacts, these examples illustrate how the 
shift to learning outcomes is making a difference 
at policy level. However, it is too early to say that 
the application of the learning outcomes approach 
has made an impact on progression between 
institutions and levels and on the accountability of 
education and training systems. 


3.4.2 The impact of learning outcomes on 
education and training management 

Research on the impact of learning outcomes on 
education and training management is limited. The 
abovementioned Cedefop study (2016 provides 
insights into the role played by learning outcomes in 


teachertraining institutions. Covering 10 institutions 
in 10 European countries made it possible for 

the study to observe the relationship (alignment) 
between the learning outcomes expressed at 
national level, at university level, within the teacher 
training faculty and in the particular programmes 

and courses delivered. Overall, the study presents 

a positive picture of the use of learning outcomes 
for curriculum and programme design. While it 

is seen by some staff members as imposing an 
extra layer of bureaucracy, a majority signalled their 
readiness to adapt and develop their programmes 
and courses in line with the approach. Interestingly, 
the main concerns were expressed in relation to the 
conceptual assumptions underpinning the proposal, 
not to the learning outcomes perspective as such. 
The report notes that while the learning outcomes 
approach informs and influences the programme and 
curriculum descriptions, the delivery of programmes 
and courses to students Is influenced to a limited 
extent only. Overall, the study illustrates the way 

in which the articulation of learning outcomes at 
different levels can be aligned to each other. While 
the influence of national (and European) level tools 
(qualifications frameworks) can, to some extent, be 
observed at the level of institutions, the translation of 
these from written documents into teaching practices 
and learning formats seems to be challenging. 


The study by Bleiklie et al. (2017) on the ambiguity 
of learning outcomes as a management tool in 
academic institutions throws some further light on 
these issues. Based on a comparative study of one 
English and one Norwegian university (focusing on 
humanities and STEM faculties in both cases), the 
study tries to capture the specific characteristics of 
learning outcomes as a management tool. Departing 
from an understanding of learning outcomes as a 
deeply ambiguous instrument, the study shows 
how different national contexts inform the actual 
application and thereby the impact of the approach. 


‘[T]he stronger the hierarchy, the more 
likely it is that LOs [learning outcomes] 
are introduced by superior levels in the 
organisational hierarchy of the institution, 
and that faculty will experience them as 
the imposition of a tool that caters for 
administrative needs. The stronger the 
disciplinary communities, the more likely 
they are to be able to shape them to be 
compatible with their preferences [...] 
The higher the degree of standardisation, 
the less involvement can be expected of 
local actors in shaping learning outcomes: 
(Bleiklie, 2017, p. 72) 


Based on these general observations, a number 
of specific characteristics of learning outcomes as 


Volume |: Thematic chapters 


39 


40 


management tools are identified. First, learning 
outcomes seem to strengthen the administrative 
information required by the university hierarchy. 
While the Norwegian faculties pointed out the 
added value of this information for students and 
as a basis for strategic planning, the English 
faculties perceived less impact. Second, learning 
outcomes seem to have influenced leadership 
roles to some extent in Norway. This is linked to 
the improvement of administrative managerial 
information. This impact was observed to a lesser 
degree in the English case. Third, the sense 

of change caused by the learning outcomes is 
stronger in the Norwegian than in the English 
case. This perception may have been caused 

by the more recent shift to learning outcomes 

in the Norwegian context, notably through 

the introduction of the NOF and a number of 
associated reforms influencing higher education. 
Fourth, when taking into consideration the 
differences between the humanities and STEM 
faculties in the two countries, the fundamental 
ambiguity of learning outcomes as a managerial 
tool is well illustrated. However, learning 
outcomes are, on the whole, considered to be a 
tool for administrative managerial information, to 
a lesser degree to be a tool for quality assurance 
and strategic planning, and only to a limited 
extent to be a tool for pedagogical reform. 


The ambiguity of learning outcomes as 
management tools is also touched upon by 
Caspersen et al. (2017a). It is pointed out that 

this ambiguity ‘[...] actually opens up a space of 
discretionary and interpretational latitude, either 
because learning outcomes are assimilated to 
traditional path dependencies or because they 
allow for the introduction of change. This ambiguity 
actually provides flexibility for contextually diverse 
implementation’. However, it is underlined that this 
reduces the level of comparability initially envisaged 
(p. 8). 


The focus on outcomes is not a trend that 

relates only to education and training, but one 
that is partly inspired by broader management 
traditions in the private as well as the public 
sector. Sharing the output orientation of MBO 
and NPM, the learning outcomes approach seeks 
to combine strategic steering from the top with 
local autonomy and adaptation. While empirical 
evidence is limited, the ambiguous character of 
learning outcomes seems to provide room for 
local interpretation and adaptation. To what extent 
this local autonomy conflicts with the ambitions 
of central authorities could be addressed by future 
research. 


3.4.3 The impact of learning outcomes on 
pedagogies 

Adam (2004; 2008) states that there is a close 
relationship between learning outcomes and 
student-centred learning. Seeing these as mutually 
dependent, he considers the move to learning 
outcomes as a paradigmatic shift from traditional 
teaching to a learnercentred approach. The 2009 
Cedefop study (p. 38) also states that ‘the use of 
learning outcomes is conducive to the emergence 
of successful policies and to the development of 
learnercentred practices in teaching and training’. 
This belief in the potential of learning outcomes 

to transform pedagogies is in line with the 
‘constructive alignment’ approach outlined by Biggs 
(Biggs, 1999; Biggs and Tang, 2007). Biggs identifies 
four main steps necessary for an alignment to take 
place: defining the intended learning outcomes; 
choosing teaching and learning activities that are 
likely to lead to these intended outcomes, and 

that help and encourage students to attain these 
outcomes; engaging students in these activities; 
and enabling learners to demonstrate what they 
have learned and/or provide formative feedback. 
‘Constructive alignment’ thus prescribes how to 
move from intended to actually achieved learning 
outcomes. 


In its study on the impact of the learning outcomes 
approach on teaching and training institutions, 
Cedefop (2016 indicates that the shift to learning 
outcomes is clearly impacting curriculum and 
programme developments. Paradoxically, these 
developments seem to influence teaching and 
learning practices only to a limited extent'®. In one 
of the institutions covered, where the learning 
outcomes approach was particularly well embedded 
in curricula, students said they had never heard 

of the approach. In another institution, where 

staff showed a deep understanding of learning 
outcomes, students were not acquainted with 

the approach and the opportunities to use it 
(Cedefop, 2016; Halasz, 2017). While admittedly 
based on a very limited set of cases, these findings 
illustrate that intended learning outcomes are not 
automatically or by default translated into new 
teaching methods. This does not mean that no 
alignment takes place; it merely hints at a lack 

of evidence and empirically based research. Few 


18 Pouliou (2014) analysed how the shift to a learning- 
outcomes-based approach affects language teaching. 
From the interview and questionnaire findings, it became 
clear that learning outcomes are intended to have a direct 
and formative impact on the curriculum of language tea- 
ching and pedagogy. Interestingly, however, the majority 
of the respondents were not familiar with the approach. 
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studies have been conducted looking at the impact 
of curriculum and programme design on teaching 
and learning, and it is not clear whether aspirations 
regarding learnercentred education and training, or, 
indeed, active and open learning, can be linked to 
the learning outcomes shift. 


Sweetman (2017) makes an effort to fill this 

gap in a comparison of Norwegian and English 
universities and the extent to which the learning 
outcomes approach has influenced teaching 

and learning practices. In line with the Cedefop 
study 2016), Sweetman’s study confirms that 
learning outcomes are, indeed, influencing course 
planning (intended learning outcomes) and, 

overall, are supporting learners by increasing the 
transparency of programmes and qualifications. 
She points out that the approach has supported 
reflection and dialogue among teachers. This has 
proved important for supporting new teachers, 

for ‘weeding out’ substandard teaching and for 
promoting a more collegial approach to the teaching 
and learning challenges at hand. While these are 
significant impacts, Sweetman is critical of the 
assertion that the shift to learning outcomes is, 

by necessity, linked to a more learner (student-) 
centred approach, as hinted at by Adam (2008), 

for example. Learnercentredness, she asserts, is 
frequently used in a casual way (Sweetman, 2017 
p. 46) and mixed up with other more or less related 
approaches (flexible learning, experiential learning, 
self-directed learning and active learning). This lack 
of clarity undermines the consistent implementation 
of learning outcomes. A learnercentred approach, 
Sweetman points out, by necessity implies a 
change in the ‘powerrelation’ between teachers 
and learners, as well as increased ‘choice’ for the 
latter. She observes few signs that such a deeper 
transformation has taken place and points out that 
the day-to-day activities of teachers are broadly 
unchanged and stable following the shift to learning 
outcomes (Sweetman, 2017, p. 52). An exception to 
this, in which the role of the learners has changed, 
is provided by group/team tasks that seek to 
integrate subject-specific and generic outcomes 
(Ibid., p. 53). 


Overall, while acknowledging the positive impact of 
learning outcomes on transparency and programme 
design, Sweetman questions the exaggerated 
claims regarding the impact of learning outcomes 
on teaching and learning practices. One possible 
explanation for this limited influence, she asserts, 
is the lack of theoretical and practical clarity around 
both learning outcomes and learnercentred 
practices. This may mean that neither is put into 
practice with the consistency required for teachers 
or students to recognise a shift (Ibid.). 


Returning to the aspirations listed above, there 
seems to be a need to adjust and clarify ambitions 
in this field. As pointed out in Cedefop (2017), the 
transformation from intended to actually achieved 
learning outcomes is of critical importance to 

the overall credibility of the learning outcomes 
approach. The lack of empirical research referred 

to above illustrates the need to further clarify 

the relationship between the learning outcomes 
approach and teaching and learning practices. Given 
that the outcomes approach will always operate (as 
one limited element) within an existing institutional, 
professional (Praitz et al., 2017) and pedagogical 
context, we need to clarify expectations. Simply 
asserting that learning outcomes, by default, trigger 
a transformation from traditional teaching to learner 
centred practices may undermine the approach 
itself. We need, instead, in line with Cedefop (2017), 
to acknowledge that learning outcomes can be 
written in many different ways, influencing teaching 
and learning in both positive and negative ways, 
pointing to opportunities as well as challenges. We 
also need to acknowledge that learning outcomes 
can be ignored by teachers and trainers. We need to 
better understand why this is the case and how this 
can be addressed. 


3.5 Measuring the impact of learning 
outcomes: Issues and the need for 
further research 


Large-scale international student and learner 
assessment schemes such as the Programme 
for International Student Assessment (PISA) and 
the Programme for the International Assessment 
of Adult Competencies (PIAAC) illustrate the 
importance of the learning outcomes perspective 
for education and training governance and reform. 
Comparing the performance of young people as 
well as adults, these schemes are seen as providing 
direct feedback on the quality and relevance of 
education and training systems. 


Ramsden (1992) states that for students, ‘the 
assessment is the curriculum’. By this he means 
that students will learn what they think they will 
be assessed on, not necessarily on the outcomes 
stated in the curriculum or programme. Thus, 
defining and describing learning-outcomes- 
based assessment criteria plays a key role in the 
overall implementation of learning outcomes. This 
point is pursued by Biggs and Tang (2007). They 
emphasise that assessments need to mirror the 
learning outcomes as described in curricula and 
programmes, not to operate in isolation. This is 
illustrated in Figure 3.1. 
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Figure 3-1. Alignment of teaching/learning and assessment to intended learning outcomes 


Teaching/learning 


Intended learning outcomes 


Assessment tasks 


the context 


Incorporate verbs that students 
have to enact as appropriate to 


Designed to generate or elicit 


The very best outcomes that 
could reasonably be expected 
containing verbs such as 
hypothesise, reflect, apply, 
relate to principle 


Mos 


Format of tasks such 


desired verbs in large classes, 
small classes, groups or individual 
activities. Such activities may be: 


= teacher-managed 
= peer-managed 


Highly satisfactorily outcomes 
containing phrases such as 
solve expected problems, 
explain complex ideas, apply to 
professional practice 


that the target verbs are 
elicited and deployed in 
context 


= self-managed 


as best suits the intended 
learning outcome 


procedures 


Quite satisfactory outcomes 
containing phrases such as 
solve basic problems, explain 
basic ideas, use standard 


Criteria specified clearly 
to allow judgement as to 
student's performance 


attempts 


Source: Biggs and Tang, 2007. 


Figure 3.1 illustrates the challenges involved in 
transforming intended learning outcome statements 
into actually achieved learning outcomes. While 
there is a vast amount of research literature in the 
area of assessment, It touches only indirectly on 
the challenge of alignment and the implications 

this has for reforming education and training. 

While Caspersen et al. (2017b) make an effort to 
review research literature in the area of ‘learning 
outcomes measurement’, they do not discuss the 
challenges relating to its alignment to intended 
learning outcomes and the teaching and learning 
processes. Instead, they point to the need to 
clarify the purpose of measurements, observing 
the difference between measuring learning 
progress (from the individuals’ point of view) and 
measuring an ‘absolute’ level of knowledge, skills 
and competence. While the former purpose is often 
addressed through self-assessment schemes, the 
latter is often addressed through traditional tests 
expressed in grades. Somewhat resembling the 
distinction between ‘norm’ and ‘criterion’ referenced 
assessments, both approaches fall short of offering 
an adequate answer to the need for appropriate 
(reliable and valid) measurements of outcomes. 


Minimally acceptable outcomes 
and applications; inadequate 
but salvageable higher level 


0 


While the learning outcomes approach is influencing 
the definition and writing of intended learning 
outcomes, the transformation of these into actually 
achieved learning outcomes is, to some extent, 
dependent on the assessment and measurement 
approaches being developed. In this context, the 
role of assessment of learning outcomes stands 
out as critical to the future reforming role of learning 
outcomes. There is need for future research to 
address this in a more systematic way”. 


The experience gained in relation to validation of 
non-formal and informal learning during the past 
few decades may prove important in this area, 
given the attention on learning-outcomes-based 
standards and the focus on batteries of assessment 
methodologies that seek to capture the diverse and 
contextually bound experiences of individuals. 


16 Leney et al. (2008), reporting on the shift to learning 
outcomes, recognise that the learning outcomes approach 
has had only a limited impact on the way learning is being 
assessed, but argue that it is only a matter of time before 
codified learning outcomes will define and control assess- 
ment practices and the curriculum in Europe. 
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3.6 Concluding remarks 


This article shows that the learning outcomes 
approach is now firmly embedded in European 
education and training policies and practices. 

The development of qualifications frameworks, 
the growing importance of validation and the 
redefinition of curricula and programme documents 
from inputs to outcomes all point to the key role 
played by learning outcomes in reforming and 
modernising education and training. Overall, major 
progress has been made in these areas over the 
past two decades. 


A key remaining question relates to the 
transformation of intended learning outcomes into 
actually achieved outcomes. While curricula and 
programmes are now, to some extent, defined 

and written according to the learning outcomes 
approach, the transformation of these intentions 
into actually achieved outcomes — or competences, 
as some might say — is a different question. The 
following key questions stand out as critical to 
future research as well as policy developments. 


e What is required for teachers and trainers to 
make active use of the learning outcomes 
approach when planning and facilitating learning? 
How can they best be supported? 

e In what ways can the learning outcomes 
approach benefit individual learners? 

e How can assessments be designed so as to 
mirror the learning outcomes approach and 
support the transformation from intended to 
actually achieved outcomes? 

e How is the alignment of standards, curricula and 
assessment assured? 

e How can curriculum and assessment policies 
work better together? 

e How can the measurement of learning outcomes 
inform the continuous revision and renewal of 
education and training? 
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CHAPTER 4. 


RVA THAT COUNTS: WHAT DATA DO WE NEED 
TO NURTURE RECOGNITION, VALIDATION AND 
ACCREDITATION OF PRIOR LEARNING? 


Ernesto Villaloa-Garcia, Cedefop, and Borhene Chakroun, UNESCO 


Summary 


More than 120 countries around the world are 
reforming their qualifications systems. Along with 
strengthening the role of qualifications frameworks, 
countries are introducing schemes that recognise 
informal learning. Recognition, validation and 
accreditation (RVA) is also at the centre of United 
Nations (UN) Sustainable Development Goal 4, 
which seeks to ‘ensure inclusive and equitable 
quality education and promote lifelong learning for 
all’, and its targets relating to vocational and tertiary 
education. The recognition of prior learning and 
certification of skills will become more important 

in the future work context, in which individuals are 
expected to change jobs and acquire new skills on a 
continuous basis. 


An obvious attraction of the recognition of informal 
and non-formal learning for policy-makers is that it 
can reduce the qualifications deficit among the adult 
population and lower the costs of reskilling and 
qualifying the low-skilled active population. Policy 
making, at both national and international levels, 
suffers huge data gaps and weaknesses in terms of 
monitoring tools for RVA schemes. Even obtaining 
basic data and ensuring its accuracy is beyond some 
systems, and the ability to disaggregate for target 
groups in order to carry out any forecasting of needs 
is out of reach for most countries. There is also a 
lack of feedback data, for example regarding labour 
market outcomes and further learning opportunities 
for RVA candidates and to inform development at 
both institutional and national levels. For national 
systems, such data is vital if policy making is to 
become more evidence-based and if attempts are to 
be made to carry out peer learning, benchmarking 
and comparison at international level. 


In light of this context, this paper will first examine 
why RVA is high on the policy agenda and what its 
policy attraction is in the context of the future of 
work and the sustainable development agenda. The 
paper will then identify key indicators that could be 
considered by decision-makers based on a critical 
review of key indicators used for monitoring and 
evaluating education and training systems and their 


relevance for RVA. These would include indicators 
on access, internal and external effectiveness 

and labour market outcomes. The paper will 
examine country case studies on France, Portugal, 
South Africa and the USA, and draw preliminary 
conclusions from promising practices. 


4.1 Introduction 


More than 120 countries around the world are 
reforming their qualifications systems. The 

reforms involve important questions about the 
most effective way to establish standards, frame 
the curriculum and assessment procedure, and 
recognise the knowledge and skills that individuals 
have already acquired in the workplace and 
community through prior learning. Along with 
strengthening the role of qualifications frameworks, 
countries are introducing schemes that recognise 
informal learning and the tremendously rich 
experience that ageing workers have acquired over 
the years. 


Recognition of prior learning is also at the centre of 
UN Sustainable Development Goal 4, which seeks 
to ‘ensure inclusive and equitable quality education 
and promote lifelong learning for all’, and its targets 
relating to vocational and tertiary education. The 
emphasis is on promoting flexible learning pathways 
in both formal and non-formal settings; enabling 
learners to accumulate and transfer credits for 
levels of achievement; recognising, validating 

and accrediting prior learning; and establishing 
appropriate bridging programmes and career 
guidance and counselling services. The recognition 
of prior learning and certification of skills will 
become more important in the future work context, 
in which individuals are expected to change jobs and 
acquire new skills on a continuous basis. 


Although there is no single international model, 
most schemes for RVA of prior learning share 

some common broad principles. They are voluntary 
processes and are based on the identification of 
learning outcomes acquired in different settings. The 
levels and types of corresponding qualifications are 
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distinguished through criteria such as knowledge, 
skills, aptitudes, attitudes and competences. 
Validation can lead to different qualifications and 
types of qualification, while assessment and 
certification processes rely on learning outcomes 
criteria rather than the traditional duration or location 
of courses. Quality assurance arrangements 

are included to support the management of 
validation processes and ensure the quality of 
their procedures. Some validation schemes 
cover all qualifications, while others relate only 
to technical and vocational education and training 
(TVET) qualifications, and in some cases only to 
occupations, such as the trade tests in several 
African and Asian countries. 


RVA is an individual process and in many countries 
is an individual right (Chakroun and Daelman, 

2018). RVA also has a public policy perspective. An 
obvious attraction of the recognition of informal and 
non-formal learning for policy-makers is that it can 
reduce the qualifications deficit among the adult 
population and lower the costs of reskilling and 
qualifying the low-skilled active population. In some 
countries, RVA is seen as an active labour market 
measure that can support insertion/reinsertion into 
the labour market. Finally, RVA is an instrument for 
inclusion and gender equality, including for migrants 
and refugees. 


Despite agreement on the importance and 
usefulness of validation, supporting evidence-based 
policy making, at both national and international 
levels, remains a challenge. There are huge data 
gaps and weaknesses in the monitoring tools 

of validation schemes. Obtaining basic data 

and ensuring its accuracy is a challenge in most 
countries, while the ability to disaggregate for target 
groups to allow for any forecasting of needs is out 
of reach for most monitoring systems. In addition, 
there is a lack of feedback and evaluation data to 
inform development at both institutional and national 
levels. For national systems, such data is vital if 
policy making is to become more evidence-based 
and if any attempts are to be made at regional and 
international cooperation and comparison. 


The thematic report Monitoring the use of validation 
of non-formal and informal learning (Duchemin, 
2016), carried out within the 2016 European 
Inventory, puts forward three main reasons for the 
lack of monitoring data on validation. 


1. There is a lack of definition and clarity on what 
validation is. In some Member States, validation 
practices consist of a series of different 
services or options, frequently with no clear 
delimitation on what constitutes and what is 
outside the scope of a validation practice. 


2. There is a great degree of fragmentation. Even 
when the delimitation of what constitutes 
validation is clear, validation can be carried 
out by many different organisations in many 
different ways and obtaining different results 
depending on the purpose. 

3. For many countries, validation is relatively new 
and monitoring systems are still in their infancy 
or have not been yet created. 


In light of this context, this paper will first examine 
why RVA is high on the policy agenda and what 
its policy attraction is in the context of the future 
of work and the sustainable development agenda. 
It will then identify key indicators that could be 
considered by decision-makers based on a critical 
review of the key indicators used for monitoring 
and evaluating education and training systems and 
their relevance for RVA. These include indicators 
on access, internal and external effectiveness, and 
labour market outcomes. The paper will examine 
country case studies from France, Portugal, 

South Africa and the USA, and draw preliminary 
conclusions from promising practices. 


4.2 The appeal for policy-makers 


RVA has been growing in importance within public 
policies. Initially confined to education and training 
technical discussions as a tool for providing flexible 
pathways in education, RVA is now increasingly 
playing a central role in discussions on combating 
social exclusion and in the creation of active labour 
market policies (Villalba-Garcia and Bjornavold, 2017). 
The emergence of the lifelong learning paradigm 
has made learning that occurs outside traditional 
educational paths as important as learning in formal 
education. Including the diverse range of learning 
activities throughout an individual's life implies 

that all learning, irrespective of where and when it 
occurs, needs to be valued and made visible. For 
some 20 years or more, in both developed and 
developing economies, RVA has been an attractive 
idea for policy-makers concerned with employment, 
education, vocational training and lifelong learning. 


Four main sets of arguments have been advanced 
for promoting policies that support RVA, namely that 
it can: 


e deliver economic and labour market benefits; 

e enhance equity and inclusion, particularly for 
low-skilled workers and workers in the informal 
sector; 

e reduce costs; 

e reduce the qualifications deficit among the adult 
population and improve pathways to further 
learning. 
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However appealing the idea might be to policy- 
makers, developing RVA schemes at scale has 
nowhere proved to be easy. RVA systems tend 

to be complex. The validation and recognition of 
skills acquired outside the formal system requires 
complex methodologies that can take into account 
the diverse nature of learning. Individuals need 

to be supported through the process, helping to 
make their skills visible and providing adequate 
proof of their learning outcomes. These proofs 

of informal learning need to be connected with 
existing standards in order to provide value to the 
certificates awarded. The output of a validation 
process needs to be usable in the labour market 

or for entering further education. Depending 

on the specific skills that are being assessed/ 
validated, a diverse range of actors will be involved. 
Supporting the documentation will require different 
professionals, while assessors of learning outcomes 
need to be trained in appropriate methodologies. 


In this context, it is clearly a complex task to create 
a set of indicators that can capture in a meaningful 
way the functioning of validation arrangements, 
relevant to support policy decisions about RVA. 


4.3 Models of RVA 


According to UNESCO Guidelines (UIL, 2012), RVA 
for all forms of learning outcomes is a practice 

that makes visible and values the full range of 
competences (knowledge, skills and attitudes) 

that individuals have obtained in various contexts 
and through various means in different phases 

of their lives. In the European context, the 2012 
Recommendation on validation (Council of the 
European Union, 2012) defines the validation of 
non-formal and informal learning as ‘a process 

of confirmation by an authorised body that an 
individual has acquired learning outcomes measured 
against a relevant standard’, and goes on to define 
four distinct phases: identification, documentation, 
assessment and certification of learning outcomes. 


Within this broad definition, countries translate 

and adapt the concept of RVA to their different 
realities and settings. Thus, validation occurs in 
many different ways, with different emphasis and 
purpose. It is possible, however, to find certain 
common principles that underlie all (or at least most) 
validation arrangements. First, validation constitutes 
a process in which the individual applicant has 
control. The principle of the ‘centrality of the 
individual’, as stated in the European guidelines 
(Cedefop, 2015), emphasises that it is the individual 
who decides to enter into the process and who 
should be able to decide when to leave it. 


Second, validation initiatives will normally include 
the four stages of validation: identification, 
documentation, assessment and certification of 
learning outcomes. Depending on the intention 

of the validation process, there will be a different 
emphasis on each of the stages. When an 
individual is aiming to obtain a formally recognised 
qualification (equivalent or the same as those 
acquired through traditional educational pathways), 
more emphasis will need to be placed on the 
assessment and certification phases. However, 
when the intention of the individual is to take stock 
of their existing skills (as a ‘bilan de compétence’ in 
France), either within a process of increasing self- 
awareness or when exploring career possibilities, 
the emphasis of the validation process will be on 
the identification and documentation phases, rather 
than the provision of very reliable certificates or 
assessment. In this second case, the amount of 
time that would need to be invested to provide 

an adequate level of assessment would not be in 
line with the individual's intentions, making the 
validation process neither efficient nor effective. 


Thus, consideration of the purpose of the validation 
system and allowing individuals to enter the process 
with different intentions are cornerstones in the 
design of validation systems. This has important 
implications for the creation of monitoring and 
evaluation systems for validation, as the data 
collected needs to be in line with the intention of 
the system in place. 


In this article we explore a number of cases in which 
data is being collected for validation. The four cases 
were selected because, in general terms, data 

on RVA is very limited. These cases, however, are 
among the few that systemically collect information 
and have had several years of collecting information 
on RVA practices. They provide interesting examples 
of how validation can be monitored. 


Table 4.1 shows the countries selected and the 
main features of their validation systems. In 
France, the validation of experience (‘validation 
des acquis de l'expérience’ (VAE)) is a procedure 
that can lead to the award of whole or parts of 
qualifications with a vocational and professional 
orientation (‘finalité professionnelle’) at all levels of 
the national qualifications framework (NOF). The 
VAE is highly focused on obtaining a certification 
and is closely connected to the national directory 
of vocational qualifications (Répertoire national 
des certifications professionnelles (RNCP)). In 
Portugal, the recognition, validation and certification 
of competences (reconhecimento, validacao e 
certificagao de competéncias (RVCC)) is carried 
out by the Qualifica centres through the evaluation 
of a portfolio by a panel that includes several 
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practitioners (Guimaraes, 2019). The process 

leads to school certification or to a professional 
certification. Since 2016, the RVCC has been 
combined with an amount of compulsory training to 
better assist candidates in the process of evaluation. 
In South Africa, recognition of prior learning (RPL) 
refers to ‘the principles and processes through 
which the prior knowledge and skills of a person 


assessment. 


Table 4-1. Main features of validation systems in selected countries 


France 


Portugal 


South Africa 


USA 


Source: Authors. 


Validation of 
experience 
(‘validation 
des acquis de 
l'expérience’ 
(VAE)) 


Recognition, 
validation and 
certification of 
competences 
(RVCC) 


Recognition of 
prior learning 
(RPL) 


Prior learning 
assessment 
(PLA) 


VAE is defined as an individual 


right in France. The purpose of 
VAE is to obtain a qualification 
(Article L. 6411-1 of the Labour 
Code). It can lead to the 
award of whole or parts of 
this qualification (certification) 
based on the knowledge and 
skills acquired through prior 
experience and learning in a 
variety of contexts (Mathou, 
2019). 


Through the RVCC, an adult can 
obtain a basic level certificate up 
to level 4 of the NOF (certificate 
of qualifications corresponding 
to the 1st, 2nd or 3rd cycle of 
basic education plus a basic 
education diploma). 


RPL is defined as ‘the principles 
and processes through which 
the prior knowledge and skills 
of a person are made visible, 
mediated and assessed for the 
purposes of alternative access 
and admission, recognition and 
certification, or further learning 
and development’ (SAQA, 2015, 
Clause 26). 


PLA is the process by which 

an individual's experiential and 
other extra-institutional learning 
is assessed and evaluated for 
the purpose of granting college 
credit, certification or advanced 
standing towards further 
education or training. 


are made visible, mediated and assessed for the 
purposes of alternative access and admission, 
recognition and certification, or further learning 
and development’ (SAQA, 2015, Clause 26). In the 
USA, RPL is a practice devolved to the individual 
institutions that can award credits towards the 
completion of a qualification based on prior learning 


Direction de l’'animation de la 


recherche, des études et des 
statistiques (DARES) 


National agency for vocational 
education and qualification 
(Agéncia nacional para 

a qualificacao e ensino 
professional (ANOEP)) 


South African Qualifications 
Authority (SAQA) 


Council for Adult and 
Experiential Learning (CAEL) 
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4.4 Dimensions for monitoring and 
evaluation of RVA 


A monitoring and evaluation framework to inform 
policy making should cover all four stages of the 
validation process: identification, documentation, 
assessment and certification. It should include data 
on how individuals enter and progress through 
RVA, how they achieve recognition and validation, 
and how they value RVA outcomes for work and 
life. A monitoring and evaluation system for RVA 
will look at: (i) pathways for entry to RVA (access 
and participation); (ii) pathways for progressing 
through RVA (quality and internal effectiveness); 
(ili) pathways for completing RVA (certification); 
and (iv) career and life pathways following RVA 
(outcomes). 


Access and participation 

This refers to the extent to which various types 

of RVA promote equity and inclusion, and the 
implications for the expansion of lifelong learning 
opportunities for excluded groups. While this 
component focuses on the important social aims 
of RVA, it is simultaneously strongly related to 

the relevance and outcomes dimension, since 

it prioritises the need to increase the number of 
people who have viable and effective opportunities 
to benefit from high-quality RVA leading to positive 
socioeconomic and labour market outcomes. 


A set of contextual indicators is required to capture 
the main drivers of RVA supply and demand, as 
well as the key factors affecting the efficiency of 
the matching process between them. Many of 
these factors will also affect the outcomes of skill 
use. Accordingly, a range of indicators is proposed 
covering five main areas: demographics, early 
childhood development, aggregate economic 
conditions, technology and work conditions, and 
education and labour market institutions and policy 
settings. 


Access indicators will allow for a better 
understanding of whether outreach measures are 
being deployed efficiently. It will be possible to 
investigate whether RVA is being used by those 
who need it the most. 


Quality and internal effectiveness 

This is a measure of the quality of any RVA 
programme, and of whether it is effectively 
conducted and relevant in terms of meeting needs. 
Ideally, it will investigate each of the stages of 

the process (identification, documentation and 


assessment) and devise indicators to capture 

how each of the stages are working in the overall 
process. Areas for these indicators would include 
resources, Support measures, intermediary outputs, 
methods and areas of study. 


It is Important to understand the resources 
allocated to the RVA process in order to be able to 
properly evaluate its efficiency and effectiveness. 
Understanding the financial resources required, as 
well as the number of people involved in recruiting, 
supporting and assessing the individual's RVA, is 
necessary if the system is to be properly monitored. 
However, these types of indicators are normally 
scarce, as in many cases financial support is not 
earmarked for validation but is included in the 
general educational budget. 


The allocation of human resources to the validation 
process should take into account the amount 

of support provided for the institutions and 
practitioners working on validation. Collecting 
information on the amount of training or the 
qualifications and credentials of the practitioners will 
help to increase trust in validation systems. 


Information on intermediary outputs of the validation 
process will also be required. It is important to 
understand the types of documentation that 
individuals are obtaining before going into a final 
assessment to gain a certification. Monitoring 
systems should ideally identify how many people 
are able to start the validation process and manage 
to obtain specific identification and documentation, 
even if they do not manage to proceed through the 
whole certification process. 


A fourth area for indicators relates to methods. 
Following consultation with US institutions, the 
Council for Adult and Experiential Learning (CAEL) 
has addressed this: ‘Perhaps the greatest need 

for standardization in the tracking of PLA [prior 
learning assessment] data is in the defining of the 
individual PLA methods that are tracked. The critical 
unit of measure is the method of PLA, which we 
define here as the specific assessment tool to be 
used’ (Klein-Collins, 2016, p. 5). The Tennessee 
Higher Education Commission, which identified 

and defined 22 individual methods of PLA, is cited. 
These different methods are grouped into five main 
general categories: standardised exams, challenge 
exams, individual assessments, evaluation of non- 
college education and training, and other (Klein- 
Collins, 2016). In addition to understanding the 
methods, it will also be important to understand the 
subject areas in which validation is occurring so that 
the resources needed can be better anticipated. 
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Certification 

This relates to the output of the validation process 
after assessment has been carried out. The 
certification phase of an RVA process might imply 
that an individual has obtained a formal qualification 
or other type of certificate. At national level, the 
system would ideally collect information in all the 
different types of certification that the RVA system 
provides, allowing for comparison of certificates 
obtained through an RVA process against those 
obtained by traditional means. In systems in which 
RVA grants credits, the monitoring system could 
collect the number of credits earned through RVA. 


The criterion most commonly adopted to 
understand the impact of RVA is the number of 
credentials (including partial qualifications or unit 
standards) awarded through this process. Some 
countries (see the French case below) take into 
account the number of individuals who engage 
in this process every year, whether they are 
successful, partially successful or unsuccessful. 


While conclusions regarding the scale of RVA are 
a sobering reminder of the extent and complexity 
of the challenges of RPL, the real question of the 
statistics is that of the denominator (Merle, 2005), 
or the number of credentials delivered each year 
through RVA. To assess whether the RVA scheme 
has been successfully established or whether it 
occupies only a marginal place in the landscape 

of adult learning, the denominator is generally the 
number of credentials delivered each year through 
RVA. In that case, RVA appears to be marginal 
compared to the mainstream of formal learning and 
acquisition of qualifications. However, the question 
then is whether we are comparing like with like. 


Outcomes 

This relates to the responsiveness of RVA to labour 
market needs and outcomes for individuals. This 
reflects the assumption that the primary and key 
role of RVA is to raise skill levels and to help to 
address skill needs at all levels in today's complex 
and changing labour markets. The outcomes of 
RVA may be difficult to assess, and data sources 
may differ depending on the expected outcomes. 
Some studies have explored the impact of RVA, 
which does allow for the collection of such data 
(Duchemin, 2016). A survey of the motivation 

for RVA in France (Céreq, 2005) highlighted four 
expected outcomes, depending on candidates’ 
motivation: 


e A protection concern: Candidates whose 
return to employment or the safeguarding of 
employment depends on the acquisition of a 


qualification, which in turn leads them to engage 
in an RVA process. 

e An integration concern: Candidates for whom 
the start of active life is marked by an instability/ 
vulnerability in employment or job insecurity. 
Candidates see RVA and the acquisition of new/ 
higher qualifications as a way to find stable work 
and better conditions of employment. 

e Areskilling concern: Candidates who are 
engaged in a perspective of professional 
reorientation. The goal is, above all, to 
obtain a level/type of qualification to satisfy 
the conditions of access for the intended 
employment. 

e Anupskilling concern: Candidates who have a 
perspective of evolution. A qualification obtained 
via RVA is expected to meet the expectations 
of social recognition and fill a gap between the 
functions performed, or the functions concerned, 
and their present qualifications. 


4.5 Case studies 


The following case studies provide concrete 
examples of indicators used for the monitoring and 
evaluation of RVA for decision-making. The cases 
cover countries with different RVA schemes and 
present common categories of indicators regarding 
access and internal effectiveness. At the same time, 
the cases illustrate diversity in the way that RVA is 
organised, its objectives and its expected outcomes. 


However, the complex nature of RVA systems and 
its fragmented nature require a very clear definition 
of what RVA is and how its outcomes will be 
measured. The increasing interlinkage between 
formal, non-formal and informal types of learning 
makes it even more difficult to collect information 
on RVA processes that are, in many instances, 
interlinked with career management processes or 
training periods. 


At the same time, the cases provide data and 
indicators beyond learning assessment data. For 
example, data on the average time taken to go 
through the scheme, or throughput rates, reflect 
the complexity of the process and the concerns not 
only regarding the capacity of candidates to engage 
and finish the process, but also the effectiveness 
of such a process compared to that of a traditional 
training pathway. 


France” 
In France, the term used is ‘validation des acquis 
de l'expérience’ (VAE). This is defined as an 


” The text for this case study is adapted from Mathou 
(2019). 
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individual right and came into force in 2002. Any 
qualification listed in the national directory of 
qualifications (Répertoire national des certifications 
professionnelles (RNCP)), can be acquired and 
accessed through VAE. The general institutional 
framework for VAE is the responsibility of the 
Ministry of Labour, in partnership with the regional 
council and the other ministries that award 
qualifications. Specific rules and practices for the 
implementation of VAE are the responsibility of each 
authority awarding the qualifications included in the 
RNCP 


Different structures and awarding bodies are 
involved in the process of organising and reporting 
on RVA. In 2014 a new law’ (following the decree 
in the same year) introduced a number of important 
changes to the VAE system, including provisions 
on the organisation of statistical monitoring of 
VAE pathways. Article 6 of this law introduced 
new provisions in the Labour Code, according to 
which the Regional Committees on Employment, 
Training and Vocational Guidance (CREFOPs) and 
the National Council for Employment, Training 

and Vocational Guidance (CNEFOP) are in charge 
of the statistical monitoring of the use of VAE. 
Furthermore, the Decree of 12 November 2014 
introduced provisions in the regulatory part of 

the Labour Code. As part of the responsibilities 
mentioned in Articles L. 6423-1 and L. 6423-2 

of the Labour Code, the CREFOPs and CNEFOP 
are now tasked with the statistical monitoring of 
VAE candidates from the beginning to the end of 
the procedure (or, in the case of partial validation, 
for as long as the credits awarded remain valid). 
Anonymous data must be submitted by VAE 
providers (both public and private organisations), 
and CNEFOP ensures the harmonisation of the 
categories of data collected to allow the monitoring 
of candidates and their pathways. 


DARES (research institute under the Ministry of 
Labour) compiles annual aggregated data from the 
various ministries involved in VAE. Qualifications 
awarded by ministries (state diploma) account for 
about 85% of all qualifications awarded through 
VAE. Data on qualifications awarded by chambers 
of commerce and trade and on professional 
qualification certificates (certificats de qualification 


8 Loi n° 2014-288 du 5 mars 2014 relative a la formation 
professionnelle, a |l’emploi et a la démocratie sociale, see 
www.legifrance.gouv.fr/affichTexte.do?cidTexte=JOR- 
FTEXT000028683576 (last accessed 15 April 2019). 
Décret n° 2014-1354 du 12 novembre 2014 portant 
diverses mesures relatives a la validation des acquis 
de l'expérience, see www.legifrance.gouv.fr/eli/de- 
cret/2014/11/12/ETSD1417561D/jo (last accessed 15 April 
2019). 


professionnelle) delivered by sectoral branches 
and employers federations through VAE are not 
centralised. Thus, DARES data does not cover the 
entire scope of the VAE'’. Nevertheless, DARES 
collects valuable information on aspects of access, 
internal effectiveness and external effectiveness. 
Data on candidates includes information on gender, 
employment status and level of qualifications. 

All statistics are aggregated by awarding body, 
qualification and subject. The number of individuals 
accepted into the VAE procedure is used to 
calculate the success rate in terms of partial or full 
validation. In some cases, it is possible to find data 
on the average duration of the validation scheme. 


In addition to providing statistics, each body 
awarding qualifications is expected to develop 

its own evaluation of the implementation of VAE. 
The law requires them to review the process and 
procedures used for VAE on a continuous basis and 
to make any necessary improvements. This culture 
of evaluation and the accessibility of data has 
allowed France to carry out different evaluations and 
plan the revision of VAE based on existing data. 


South Africa 

In South Africa, the term ‘recognition of prior 
learning’ (RPL) is used. The South African 
Qualifications Authority (SAQA) is responsible for 
collecting and producing data on RPL. SAQA data 
(see Table 4.4) shows that a total of 23 260 learners 
on the National Learners» Records Database (NLRD) 
have achieved qualifications through RPL between 
2006 and 2015. This is considered to be the number 
of successful RPL candidates in this period. The 
total number of records for the achievement of 
qualifications through RPL in the same period was 
25 631, while the number of part-qualifications was 
1 067 802. 


SAOA data also covers sector focus and level of 
qualifications. From a sector point of view, RPL 
achievements are highest in the services field, 
followed by business, commerce and management 
studies and manufacturing, engineering and 
technology. There were no RPL achievements in 
the fields of education, training and development; 
law, military science and security; or physical, 
mathematical, computer and life sciences. 


19 For instance, the following, in particular, should be 
added to the data published in 2014 by DARES (25 300 
validations for ministerial certifications): 1 500 VAE, with 
approximately 110 issued by private certifiers; and 4 300 
professional qualification certificates, including a little less 
than 50% of the metallurgy branch (IGAS-IGAENR, 2016). 
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Table 4-2. Validation rate of qualifications submitted for the various awarding ministries, 2015 


NOMINATED | VALIDATED | PARTIAL TOTAL PARTIAL 
[1] [2] VALIDATION | VALIDATION | VALIDATION 
[3] ay = 2)) ay = 2)) 
[2/1] [3/1] 


Ministry of Higher 4070 2 589 1 064 63.6 26.1 
Education and 
Research 


Ministry of Health 12 141 4 965 52s 
and Social Affairs 


Ministry of Defence 


Ministry of Ecology 


All certifying 41 415 24 642 iss 
ministries (excluding 
Ministry of Culture) 


N/A: data not available. * In 2015, the calculation method for the number of nominated, validated and partial validation 
candidates at the Ministry of Employment changed. ** Estimated figure. 


Note: In 2015, of the 19 324 candidates who appeared before a VAE panel of the Ministry of Education, 13 153 (68.1%) 
obtained a complete validation of their title or diploma, 3 946 (20.4%) obtained a partial validation, and 2 225 (11.5%) did not 
obtain validation. Field of study: France. 


Sources: Certifying ministries; analysis conducted by DARES. 
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Table 4-3. Percentage of candidates who obtained a qualification through VAE within the TVET system, 
2015 (continuing training) 


QUALIFICATION NUMBER OF NUMBER OF TOTAL NUMBER | PERCENTAGE 
CANDIDATES CANDIDATES OF DIPLOMAS (e) 5 
WHO OBTAINED A WHO OBTAINED CANDIDATES 
DIPLOMA THROUGH | OBTAINED OUTSIDE WHO 
CONTINUING A DIPLOMA INITIAL GRADUATED 
EDUCATION THROUGH VAE | TRAINING THROUGH VAE 
(INCLUDING [2] ee | [2/3] (%) 
INDIVIDUAL 
CANDIDATES) 
[1] 
_ Brevet de 22 790° 4 395 27 185 16 
| technicien 
supérieur (BTS) 
which corresponds 
to high technician — 
Professional 6 245 2 eG} 8 798 29 
Bachelor 
Professional 3936. 619 4 555 14 
| certificate 
Certificat 54 583 3 190 ar 77S 6 
d’aptitude 
professionnelle 
(CAP) 
Brevet d'études 7326) 114 7 440 2 
professionnelles 
-(BEP) | | | 
Additional mention 1 387 199 1 586 13 


(level IV and V of 
qualifications) 


Total 96 267. 11 264 107 337° 10° 
Note: In 2015, 29% of candidates who obtained a professional Bachelor's qualification after their initial studies obtained it 
through VAE. Field of study: France. 


Sources: Department of Evaluation, Foresight and Performance. 


Table 4-4, Summary of RPL data in South Africa, 2006-15 


se SUMMARY RPL DATA TOTAL 


| Learners Number of learners who achieved one or more qualifications via RPL | 23 260 
Number of learners who achieved one or more unit standards via RPL 45 806 
Records of Number of records of achievement of qualifications through RPL 25 631 


achievement 
Number of records of achievement of part-qualifications through RPL 1 067 802 


Qualifications —= Number of qualifications achieved through RPL 178 


Source: SAQA. 
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Figure 4-1. Learner achievements through RPL by NQF field 


Services 

Business, Commerce and Management Studies 
Manufacturing, Engineering and Technology 
Physical Planning and Construction 
Agriculture and Nature Conservation 
Undefined 

Culture and Arts 

Human and Social Studies 

Communication Studies and Language 


Health Science and Social Services 


Source: SAOA. 


Portugal 

In Portugal the term ‘reconhecimento, validacao 
e certificacao de competéncias’ (RVCC) is used. 
Portugal's historical deficit in education implies 
that many of Its adult citizens are considered low 
qualified. In fact, despite the remarkable progress 
made over the past decade, more than half of 
Portuguese adults aged between 25 and 64 have 
not completed upper secondary education. 


The national system of prior learning assessment 
and recognition allows adults’ skills to be 
recognised, so that they can obtain qualifications 

at the primary and secondary education level. 

The government's New Opportunities Initiative 
(Iniciativa Novas Oportunidades (NOI)), which ran 
from 2005 to 2013, helped a large number of adults 
to participate either in processes that recognised 
their prior learning (RVCC) or in training. Due political 
changes, NOI was discontinued and replaced with 
the network of OQualifica centres, officially launched 
in 2017, According to the National Education Council 
(Conselho Nacional de Educagao (CNE)) (2011, 

p. 138), some 1 316 955 adults were enrolled at 
centres between 2000 and 2010 in order to obtain 
basic or secondary certification. According to the 
OECD (2018), by 2010 over 500 000 adults had 
participated in some form of adult learning through 
NOI, the majority by completing RVCC. The current 
number of enrolments is significantly lower, though 
it is progressively increasing (Guimaraes, 2019, and 
Figure 4.2). 


The national agency for vocational education and 
qualification (Agéncia nacional para a qualificagao e 


11,119 


ensino profissional (ANOEP)) collects data on RVCC. 
As part of the monitoring and quality assurance 
mechanisms, there is a restricted platform through 
which the validation centres register progress. The 
agency produces a monthly quantitative report 

with information relating to enrolments, referrals 
and certification in RVCC. It also provides quarterly 
reports on activities carried out by the five regional 
monitoring teams in which challenges are identified. 
Finally, an annual report assesses the functioning of 
the network of Qualifica centres and presents the 
results of the self-evaluation that each centre carries 
out. Quantitative data is collected on participation, 
achievement and success rates and on the length 
of the procedure. According to existing legislation, 
annual reports on quality assurance referring to 
RVCC (of school and professional certification) 
should be made public, but as the initiative is 
relatively new, these reports are not yet in the public 
domain (Guimaraes, 2019). 


USA 

In the USA, RVA is referred to as PLA. The 
processes are decentralised to the level of 
institutions that can provide award credits based 
on competence assessments. CAEL is a non-profit 
organisation that supports the RVA process by 
working with adult education providers, employers, 
and cities, states and regions. It has carried out 
several studies and data-collection exercises on 
the PLA process. Of particular interest is CAEUs 
examination of the academic records of adult 
learners and its comparison of outcomes for 
students who have used PLA to obtain credits with 
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Figure 4-2. Number of people enrolled in the RVCC process, 2006-17 
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Source: Xufre and Duarte (2018). 
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Figure 4-3. Certifications obtained via the RVCC per year, 2006-17 
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Source: Xufre and Duarte (2018). 


students who have earned credit through traditional 
means. The research also differentiates between 
different methods of PLA. 


Figure 4.4 shows the percentage of individuals 
who were awarded a credential by obtaining credits 
through PLA and those who were not. Interestingly, 
the data shows that students who go through PLA 
seem more motivated to finish the course than 
those who do not. The data allows for a type of 


9,133 


4821 


co 


2a) 298 Pic i oh 


2011 2012 2013 «-2014-«2015-«« 2016 ~—-2017 


@ Partial certification in RVCC 


analysis that is not normally possible in most of the 
existing monitoring systems. 


Figure 4.5 shows more detailed data collected by 
CAEL on the methods used to earn PLA credits and 
the level of completeness of degrees. This detailed 
level of analysis permits a better understanding 

of the effect of different methods for validation. 
Combined with sociodemographic variables, it 
should be possible to evaluate the extent to which 
certain methods benefit less advantaged individuals. 
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Figure 4-4, Percentage of people earning PLA credits 
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Source: Klein-Collins (2016), based on Klein-Collins (2010, p. 4). Figures based on a review of more than 60 000 students in 48 


US institutions. 


Figure 4-5. Degree completion by PLA credit-earning methods for all students 
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4.5 Looking to the future: Education 
management information systems 
and learners’ records — choices, 
opportunities and constraints 


In most countries, education management 
information systems (EMSs) are the tools used 
to gather, process and interpret data on education 
and learning. For the past several decades, these 
systems have grown in complexity to give both 
policy-makers and classroom teachers a view of 
whether learners, schools and national education 
plans are progressing in relation to different 
objectives (UNESCO, 2018). 


While all countries collect information on students, 
and qualification completion rates are important 
indicators in national statistics, RVA information is 
not usually included in the EMIS. 


Educational institutions and awarding bodies usually 
have a lot of information on learners. Learners’ 
records are rich and tend to have individuals’ basic 
demographic information. 


It would be relatively easy to include within these 
learning records specific information that would 
permit a better understanding of the use of RVA 
processes. The cases presented in this article show 
how different countries provide a series of limited 
but important indicators that could constitute 
interesting examples or starting points for a more 
complex collection of data. 


According to UNESCO (2018), numerous countries 
have started linking learners’ educational records 
with data from other government agencies, covering 
human services including those relating to family 
welfare, employment status and social protection. 
These integrated systems can yield powerful 
insights that promote a more holistic understanding 
of a particular learner's experiences. It is then 
possible to speak of ‘integrated data systems’, 
allowing ‘linkage of administrative data from 
multiple government agencies’ (US Department of 
Education, 2017, p. 1). 


Countries will need to consider carefully the privacy 
implications as they move towards a better use 

of learners’ records for decision-making. Not only 
are the solutions regulatory, but they also extend 

to technology itself. Blockchain solutions are often 
cited as a possible tool for enhancing privacy and 
security concerns surrounding learning credentials 
and other educational records (Keevy and Chakroun, 
2018). Because of the immutability of entries and 
absence of a centralised information warehouse, 
this technology has the potential to give individuals 
greater control of their learning data. Experiments 


with blockchain credentialing have been carried 
out in a number of countries (Keevy and Chakroun, 
2018; Grech and Camilleri, 2017). 


With regard to the trend to integrate disparate 
datasets, the world has witnessed a growth in 
longitudinal data systems. Longitudinal studies are 
those in which data is collected at specific intervals 
over a long period in order to measure changes 
over time. This practice is particularly advanced in 
Australia”°, France?', the USA”? and other countries. 
However, such surveys do not provide information 
on RVA graduates’ learning and career pathways. 


In France, several longitudinal surveys specifically 
targeting RVA graduates have been conducted. For 
example, DARES and Céreg, together with other 
agencies, conducted a longitudinal study in 2007 
regarding individuals who had engaged in RVA in 
20057°. The questionnaire was structured into 16 
modules. The first five modules were intended to 
describe the candidate's profile and situation at the 
time he or she deposited the admissibility booklet 
(level of training, situation in relation to employment 
(where appropriate), situation in relation to job 
search, experience in terms of vocational training, 
professional experience). The next nine modules 
were intended to describe the candidate’s journey 
from the outset to the date of the survey (initiative, 
current situation, possible interruption of the 
process, actions taken after partial validation, forms 
of assistance given to the candidate, etc.). The aim 
of the penultimate module was to examine the 
individual's professional situation at the date of the 
investigation in order to compare it to that at the 
start of the process. Another survey conducted by 
BVA as part of the IGAS-IGAENR report (2016) found 
that of the 81% of VAE users who were Satisfied 
with the process, 17% reported a positive change 
in their situation (promotion or career evolution, 
9%; finding a job, 5%; career change, 3%; personal 
achievement, 16%) (Mathou, 2019). 


Other international studies have explored 
participants’ experiences (Carneiro, 2011; Sandberg, 
2011; Stendlund, 2012 and their labour market 
outcomes (Lima, 2012). In general terms, RVA 

is reported to have had a positive impact on 
individuals’ self-stem and motivation for further 
education. 


Efforts to integrate previously walled-off data 
to reveal insights about the delivery of social, 


20 See NCVER (2018). 

21 See Gaubert et al. (2017). 
22 See NCES (n.d.). 

23 See Dares (n.d.). 
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educational and training services are primed to 
accelerate. 


Any information system for RVA would necessarily 
need to take into account the following aspects: 


1. a clear definition and delimitation, as well as the 
objectives, of the RVA process; 

2. acoordinating/integrated data-collection process 
across ministries (e.g. single request); 

3. submission through a single, simple digital 
system for all the providers that collects 
information specifically on RVA; 

4. a reduction in the data requirements, focusing 
only on indicators relevant to the performance 
monitoring and evaluation of the RVA system. 
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CHAPTER 5. 


RECOGNISING PRIOR LEARNING EXPERIENCES 
OF MIGRANTS AND REFUGEES FOR INCLUSION 


Ruud Duvekot, independent consultant, and Raul Valdés-Cotera, UNESCO Institute 


for Lifelong Learning 


Summary 


Learning throughout life is important for everyone, 
whether they are individuals, organisations, 
schools or institutions. Linking learning to social 
inclusion and career opportunities, and the changes 
in prevailing attitudes on learning, are important 
themes that allow us to understand the current 
transition towards a lifelong learning society. This is 
particularly pertinent to refugees and migrants and 
their strive for inclusion in a new country. 


The recognition, validation and accreditation 

(RVA) of non-formal and informal learning and the 
valuable role it can play in helping refugees and 
migrants to engage with their host societies is at 
the heart of this chapter. RVA is presented in its 
process-oriented framework, covering the roles and 
responsibilities of the main stakeholders: refugees 
and migrants, schools and universities, authorities, 
non-governmental organisations (NGOs), employers, 
trade unions, civil society organisations, etc. The 
objective is to demonstrate how and where to set 
up interventions for strengthening RVA in order to 
develop effective and tailored learning strategies 
that will benefit all stakeholders and, most of all, the 
refugees and migrants seeking inclusion in a new 
country. 


5.1 Introduction 


The number of refugees and migrants (newcomers) 
across the globe who are escaping wars or critical 
life conditions and looking for new life opportunities 
has increased dramatically in recent years and is 
likely to continue to grow. There are an estimated 
244 million international migrants globally (UN, 
2017), with work being the main reason for people 
to migrate internationally. In addition, refugees 
constitute a considerable number of those who 
have moved: there are almost 70 million people 
worldwide who have been forced from their homes; 
almost 20 million of them have fled their country, 
including 6.3 million Syrians. Furthermore, about 

40 million people are internally displaced, that is, 


they have moved within their own country (VWN, 
2018; UNESCO, 2018). 


Newcomers face many challenges in settling into a 
new country, in particular obstacles to accessing the 
labour market or continuing their studies, alongside 
linguistic and cultural barriers. There is a need to 
offer flexible and personalised learning pathways 

to create favourable conditions for social inclusion. 
Valuing and recognising prior (formal, informal and 
non-formal) learning, and skills and competences of 
refugees and migrants as a means of empowering 
and including them in their new country is at the 
core of these pathways. 


RVA of non-formal and informal learning outcomes 
is key to realising participative opportunities in a 
new country, for example, obtaining a qualification, 
seeking employment, and escaping poverty and 
social exclusion. Of course, such pathways to 
empowerment and employability depend on many 
factors, such as prevailing social and economic 
arrangements and the acknowledgement of 
political and civic rights, as well as opportunities 
for education and training at all levels and across 
all sectors of society: at work, in formal education/ 
training and in the community. 


A major challenge faced by newcomers is that 
although they are often educated and skilled, their 
competences might not be recognised in the new 
country. Their skills and knowledge might not fit into 
predefined bureaucratic policies and procedures, 
documentation might be lacking, or their initial 
studies might not match the qualification structures 
in the new country. This hinders their access to 

the labour market and their ability to continue 

their studies, thus jeopardising their chances of 
integrating. Moreover, in only a small number of 
countries are newcomers a main target group 

for RVA, although it is quite obvious that the RVA 
process offers opportunities for newcomers to 
create chances for inclusion (Singh, 2015; Souto- 
Otero and Villalba-Garcia, 2015), irrespective of the 
kind of education and training they received in their 
home country and the type of qualifications already 
obtained. 


This chapter analyses the state of the art of RVA 
in various world regions and recommends ways 

in which RVA for refugees and migrants can be 
implemented more effectively by recognising their 
prior learning and creating favourable pathways for 
inclusion in the new country. 


5.2 Impact of migration and 
displacement on education systems 


One of the main roles of national education systems 
is to prepare young people for participation in 
society and to support adults so that they continue 
to learn throughout their lives. A high influx of 
newcomers puts a lot of pressure on these systems 
in terms of access to education, summative/ 
formative assessment and the availability of tailored 
learning pathways. 


5.2.1 Access to education 

It is vital for people to have access — or instrumental 
freedom - to all forms and phases of learning in 
order to shape their own destinies (Sen, 1999). 
Enabling migrants and refugees, especially adults, 
to access education is crucial for promoting 
inclusion. However, issues such as language and 
cultural differences, missing certification, and 
funding can impede newcomers’ ability to benefit 
from national education systems. Many countries 
exclude refugees and migrants from their national 
education system. Asylum-seeking children in 
detention in countries such as Australia, Hungary, 
Indonesia, Malaysia and Mexico are given little 

or no access to education. Rohingya refugees 

in Bangladesh, Burundian refugees in Tanzania, 
Karen refugees in Thailand and Afghan refugees in 
Pakistan can obtain an education only in separate, 
non-formal, community-based or private schools, 
some of which are not accredited. 


In contrast, with the 2009 Right to Education 

Act, local authorities in India are legally obliged to 
admit migrant children, while national guidelines 
recommend flexible admission, seasonal hostels, 
transport support, mobile education volunteers 

and improved coordination between states and 
districts (UNESCO, 2018). In other countries, 
integration programmes promote the acquisition of 
the local language and cultural habits. In Lebanon, 
for example, where the government and its 
partners developed the Reaching All Children with 
Education (RACE) programme for Syrian refugees, 
educational assistance and alternative learning 
programmes have benefited close to 50% of those 
requiring basic education (Singh, 2018). Meanwhile, 
in Sweden, information and communication 
technology (ICT) in the form of audiovisual methods 


is being used to help migrants develop their 
language proficiency faster, with ICT applications 
within and outside the classroom providing 
opportunities for independent learning. Similarly, 
Germany launched a smartphone app comprising a 
basic German language course, information on the 
asylum application process and on how to find jobs 
and training, and information on German values and 
social customs in order to help migrants and asylum 
seekers to integrate into the country (IOM, 2018). 
Finally, in the Netherlands, volunteers are deployed 
as language coaches to help refugees develop their 
skills (VWN, 2014). 


5.2.2 Assessing learning needs 

RVA of non-formal and informal learning converts 
prior learning experiences into new learning 
opportunities, not only in terms of access to 
education and qualifications, but also in terms of 
awareness and articulation of previous learning 
achievements on which further education can be 
built. Newcomers need to be able to articulate their 
learning needs, identify personal competences and 
gain an overview of social perspectives in their host 
societies. However, practice shows that educational 
standards are hard to calibrate with individual 
learning needs, particularly when it comes to linking 
the prior learning experiences of a learner and the 
learning outcomes of qualifications or occupational 
standards. 


Refugees need guidance and counselling in order 
to have their prior learning, skills and experiences 
recognised and to help them understand the 
different education and training options and 
opportunities available to them (Singh, 2018). In 
Indonesia, India, Brazil and Mexico, this type of 
guidance and counselling is provided by national 
agencies appointed by central or local governments, 
whereas in other countries it is managed by NGOs 
and schools/universities (Branika, 2016; UIL, 2016). 
A positive example of the latter can be found in 
the Netherlands, where the International Women's 
Centre (Internationaal Vrouwen Centrum (IVC)) 
contributes to the emancipation, participation and 
integration of migrant women by offering training 
on the self-management of competences. Migrant 
women are encouraged to describe and document 
their personal skills and competences to increase 
their ownership of their careers and livelihoods 
(Duvekot, 2016). 


During assessment, formative and summative 
monitoring methods are used to link competences 
with qualifications and occupational standards. 
Learners can assist in this process with self-tests 
and portfolios, which describe, document and 
reflect their prior learning. Assessors compare these 
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personal competences with a selected standard 
for the intended learning objective; this, in turn, 
provides a basis for drawing up advice on access to 
education and training, accreditation and concrete 
further learning steps. 


Many countries integrate such assessment 
services into their national education systems, 
while others offer assessment in independent 
centres. In Indonesia, for example, assessment 
services for low-skilled workers are embedded 

in national education and training programmes: 

the Indonesian Government works together with 
schools to develop guidance, assessment and 
(further) learning trajectories (Hasbi, 2016). In South 
Africa, sectorbased assessment centres have been 
established based on local needs (McKay, 2016). In 
Norway, ‘Skills Norway’ acts as a national centre for 
assessing and training adult immigrants in language 
skills and social studies (MJPS, 2016). 


5.2.3 The need for alternative pathways 

In the New York Declaration for Refugees and 
Migrants (UN, 2016), adopted by the UN General 
Assembly on 19 September 2016, Member States 
pledged to ‘expand the number and range of legal 
pathways available for refugees to be admitted to or 
resettled in third countries’. These pathways were 
broadly divided into three main streams: labour 
mobility, education pathways and family migration. 


RVA answers this need for alternative, or 
‘personalised’, pathways by providing access 

to learning and enhancing career opportunities. 
Furthermore, RVA allows newcomers to take an 
active role in achieving personal, civil and social 
‘effects’. A ‘civil effect’ means achieving a learning 
outcome in the context of a qualification standard 
within the education system. A ‘social effect’ 
focuses on results that are relevant to job profiles, 
targets, participation goals or assignments, as 

well as the personal impact of empowerment 

and personal development. Each pathway and 
recognition mechanism should encompass this wide 
range of effects. For example, the Australian Human 
Rights Commission has proposed several measures 
to enhance study opportunities for refugees. These 
measures include providing loans for tuition fees or 
offering domestic tuition rates to refugee students, 
allowing applicants to bring members of their 
family into the country and providing targeted study 
opportunities that are linked to Australia’s skilled 
migration needs (AHRC, 2016). 


5.3 Ensuring flexible learning and 
support 


Flexible learning pathways and support mechanisms 
enable newcomers to move within and across 
education, training and employment opportunities. 
Flexibility means that they can adapt their pathway 
as they move forward and keep a close eye on 
their interests and abilities in their learning process. 
In systems that lack flexibility, it is difficult for 
newcomers to access education and jobs and to 
make the transition from one pathway to another. 
They may find that they are restricted to their 
original choice, even if they have realised that this 
choice is no longer right for them. It is therefore 
important to provide flexible pathways as second- 
chance opportunities for young people and adults 
to enter the education system and the world of 
work. This entails (i) recognising already acquired 
non-formal learning outcomes in addition to early 
formal education; and (ii) Supporting mechanisms for 
documentation, guidance, assessment and tailored 
learning, and for anchoring learning outcomes to 
social achievements such as qualifications, job 
enrolment and citizenship. 


Ensuring flexibility and support mechanisms in 
learning pathways involves setting up an interlinked 
and integrated process (see Figure 5.1), ideally of 
four phases: 


e outreach, in which newcomers are approached 
and their learning needs are articulated, resulting 
in the determination and preparation of a desired 
and appropriate learning path; 

e assessment, which helps to determine a 
personal learning trajectory, including what has 
or has not already been learned; 

e¢ a tailored learning programme, in which the 
learning trajectory Is implemented in accordance 
with what, when and how the expected learning 
outcomes can be reached; 

e validation, in which the learning path is 
maintained and what has been learned is 
practised; as such, validation is important for 
emphasising learning as a lifelong learning 
process. 


RVA is an area for reform in many of the countries 
that are seeking to provide flexible pathways for 
newcomers. The development of these pathways 
and support mechanisms may depend on contextual 
features and feature a variety of approaches. 
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Figure 5-1. RVA-steered, personalised learning pathways 


Outreach 
Information 
Matching & intake 
Workshops 


Assessment 
Test instruments 
Interviews 

Work visits 
Formative advice 
Summative outcomes 


Counselling & guidance 
Prior learning 
Portfolio training 


Tailored learning 
& working 
Blended learning 
Work-hased learning 
E-learning 

Learning teams 

Modular learning 
Formative feedback 


Guidance, tutoring & 
mentoring 


Validation 
Qualification 
Hybrid careers 
Lifelong learning 


Source: Authors. See also: https://ec.europa.eu/social/main.jsp?catId=1224 


However, in the end, the primary goal is to ensure 
that RVA: 


is embedded in national and/or regional- 
level laws, creating a legal framework for 
designing, applying and evaluating RVA - 
The EU has already set up such a framework: 
Upskilling Pathways is a relatively new initiative 
targeting adults with low-level skills who may 

be in employment, unemployed or economically 
inactive. European Member States may 

define priority target groups for this initiative, 
depending on national circumstances (Council of 
the EU, 2016). 

links prior learning outcomes to national 
qualifications by a process of identification, 
documentation, assessment and certification 
— Singh (2018) reports that Egypt, Jordan, 
Lebanon and Turkey are developing national 
qualifications frameworks (NOFs) in order to 
recognise qualifications based on learning 
outcomes and competences acquired in all 
settings: formal, non-formal and informal. 

NOFs not only reflect expected learning 
outcomes, but also inspire curriculum design 
and assessment criteria as well as teaching and 
training methods. 

is offered within the labour market by 
partners (employers, trade unions) who 
facilitate the identification, documentation, 
assessment and certification of prior 
learning outcomes — The outcomes can be 


geared towards finding a new or better/higher 
paid job. Such labourmarket-steered learning 
pathways are available in countries such as 
Germany, the USA and Australia (Schuster 

et al., 2013). 


e links prior learning experiences to civil 


society perspectives — Employers and 
voluntary organisations need to develop 
mechanisms for recognising individual 
competences to allow refugees to learn while 
they work in both voluntary and paid jobs. 
Studies in the Netherlands (Duvekot, 2016) and 
Australia (Webb, 2015) show that work-based 
learning opportunities benefit newcomers in 
terms of their labour market orientation and 
social integration. In New Zealand, one of the 
recognition frameworks targets adults who 
have the necessary capacities and personal 
skills to work as community practitioners and 
to facilitate civil society initiatives (MoE, 2008; 
Branka, 2016). 


e is sufficiently funded — National governments 


currently fund the majority of RVA for 
newcomers; however, NGOs and schools/ 
universities also have a part to play. A case 
study of RVA for refugees in European higher 
education shows that the chance of successful 
inclusion rises when sufficient funding is 
available (Duvekot and Duvekot, 2018). 
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5.4 The importance of 
multi-stakeholder partnerships 


Because RVA links various stakeholders — schools/ 
universities, ministries, employers, trade unions, 
NGOs, etc. — managing these partnerships is 
imperative for the creation of a more inclusive 
education system for refugees and migrants. 

RVA of prior learning is a key component of 

lifelong learning, particularly for refugees and 
migrants who are trying to build a career in a 

new country. Strengthening RVA practices entails 
raising awareness of the value of prior learning 
experiences, providing information on RVA services 
(information, guidance, assessment), strengthening 
the linkage with qualification standards and 
occupational standards and coordinating among 
national stakeholders and schools/universities 
(Duvekot and Duvekot, 2018). 


Successfully applying RVA through the upscaling 
and linkage of qualification/sector standards 
creates social perspectives for newcomers that 
can be linked to their motivation/ambition (agency) 
and their intrinsic values and developmental 
potential. Such outcomes are reported in, for 
instance, Norway, the Netherlands, Switzerland 
and Denmark (Cedefop, 2017). This impact is 
also demonstrated in the case of Syrian refugees 
in Jordan and Lebanon (Singh, 2018). Bray and 
McClaskey (2015) perceive a focus on utilising 
national or sector frameworks for the purpose of 
personalising the learner's potential as the next 
development in the learning system based on 
the engaged, self-managed, learning individual 


operating in a Supportive social network. The offer 
of a shortened, self-steered and flexible learning 
path for obtaining educational levels is, in this 
respect, more than just a formalising of personal 
learning experiences; rather, it is an offer of 
personal empowerment in terms of strengthening 
life skills, including literacy and numeracy, and 
social participation in the learning society. 


A critical success factor in achieving a specific 
outcome of RVA for competent refugees and 
migrants is to offer a social perspective. Sector 
based and regional RVA practices produce 
successful outcomes (Singh, 2015; Loo, 2016), 
especially if the organisations and companies 
involved ensure that the formulation of their 
demands for functions and activities at specific 
qualification levels is transparent. This strategy 
works especially well when focusing on tackling 
skills gaps or offering further learning paths for 
target groups such as newcomers. Furthermore, 
as a follow-up on RVA processes, schools and 
universities need to offer tailored further learning 
options. These need to be flexible in order to boost 
individuals’ desire to continue learning. The options 
should address not only the ‘what’ of the learning, 
but also the ‘how’ in terms of (combinations 

of) work-based learning, classroom instruction, 
mentoring/tutoring, self-steered learning, distance 
learning, and so on. 


RVA helps to create an integral complex of shared 
responsibilities among the main stakeholders: 
learners, organisations on/around the labour 
market and schools/universities. These three main 


Table 5-1. Drivers, impacts and geographical occurrences of RVA for refugees and migrants 


DRIVER 


IMPACT ON INCLUSION 


Social 


re-integration into social life 


Motivation, self-management of competences 
(empowerment), redressing past inequalities, 


GEOGRAPHICAL OCCURRENCE 


South Africa, Indonesia, India, 
Norway, Uruguay, Netherlands, 
Switzerland 


Participative 


awareness 


Sources: Bratka, 2016; Duvekot and Duvekot, 2018; Singh, 2018. 


Civil society, social activation, volunteering, 


Netherlands, Scandinavia, South 
Africa, Canada 
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stakeholders are supported indirectly, or even 
directly, at a macro level in society by national, 
regional or local authorities and social partners. 
Table 5.1 shows how the coordination of such 
integrated RVA can be set up, regardless of the 
driver — economic, social, educational, participative 
— that prevails in a national context where RVA 
systematics are embedded in a multi-stakeholder 
approach. 


5.5 Challenges and policy 
recommendations 


Strengthening the role of RVA for enhancing 

the inclusion of refugees and migrants needs 
coordinated action and, above all, open dialogues 
between learner and teacher/employer in the two 
interlinked processes of validation and learning. 


e In the validation process, the testing 
(Summative) and advisory (formative) functions 
of validation in the dialogue between (the 
portfolio of) the learner and (the qualification 
standards of) the teacher and/or (occupational 
standards of) employers are used to recognise, 
validate and accredit learning outcomes. The 
learner is assisted in this process by test 
forms that are product- or process-oriented. 
Process-oriented means that learning outcomes 
are assessed in an integrated and interlinked 
manner, whereas in product-oriented validation, 
assessment is based on the value of the learning 
outcome itself. 

e The (further) learning process aims to offer 
learning that cannot (yet) be assessed through 
validation or learning that the learner believes is 
desirable. This process is personalised in terms 
of content, form and meaning. The learner can 
have a say in the design of the personal learning 
trajectory within the framework of the agreed 
learning trajectory. 


RVA approaches for newcomers can enhance the 
dialogues on utilising newcomers’ learning potential 
in validation and learning processes for social 
inclusion. The aim of analysing the challenges and 
formulating policy recommendations is to create 
favourable conditions for such dialogues. 


5.5.1 Keep it simple and informative 

A major challenge is avoiding complexity in the RVA 
process and offering well-organised information 
and guidance services. A Swiss case study relating 
to migrant women (Bednarz and Bednarz, 2014) 
highlights three success factors, namely that RVA 
is a means to (i) enable individual empowerment by 
making prior learning visible; (ii) enable individuals 


to achieve a qualification and thereafter enter 

the workforce (employability); and (iii) enable 
individuals to access higher education. However, 
these successful outcomes were realised only by 
consciously avoiding complex procedures and by 
offering plenty of information and clear guidance. 
What also proved to be beneficial was a high level 
of awareness that familiarisation with local language 
and norms needs to be part of validation logistics 

in general. In addition, in countries such as Jordan, 
Lebanon and Indonesia there is an awareness of the 
importance of accessible procedures, the provision 
of information, and low-complexity and guided 
process steps in RVA (Singh, 2018; Hasbi, 2016). 


The provision of information and guidance in the 
RVA process and the subsequent availability of 
counselling and guidance services play an important 
role in the utilisation of RVA by newcomers. 
Information and guidance are the so-called ‘front 
office’ for the recognition of individuals’ prior 
learning experiences in terms of education levels 
(Aggarwal, 2015; Cedefop, 2017). In order to deliver 
information provision and guidance properly, staff 
members of NGOs, schools and universities, RVA 
centres, etc., not only need to be well trained 

n guiding and counselling individuals, but also 
need to be able to oversee the whole process of 
RVA, including the further learning programming. 
This expertise extends to the responsibility 

of providing adequate information and raising 
awareness of the potential value of an individual's 
learning achievements in terms of access to a 
school or university. Furthermore, staff members 
need to perceive RVA as an open process that 
depends largely on the internal policy of schools/ 
universities on (i) facilitating RVA not only for formal 
learning, but also for non-formal and informal 
learning experiences; and (ii) the creation of an 
open dialogue between newcomers with personal 
learning experiences and schools/universities with 
structured qualification standards. RVA needs to 

be ‘easy’, well guided, and tuned in to progression 
to further learning and transition into the world of 
work. 


5.5.2 It is a right 

Many countries grant access to the education 
system based on an individual's right to attain 

basic education levels. Alongside this, access to 
portfolio guidance and assessment can be seen in 
terms of an individual's right to integrate in the new 
country not just for access to education but also for 
social guidance and counselling, capturing both the 
retrospective value of learning (i.e. the assessment 
of prior learning experiences) and the prospective 
value of further learning (such as learning pathways 
tailored to personal learning style and contexts). 
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In France, the legal basis for RVA gives individuals 
the right to have their formally, informally and 
non-formally acquired experiences assessed. 
Such a right also exists in Canada, Iceland 

and Norway, where individuals can have their 
non-formal and informal learning assessed 
(Gouvernement du Québec, 2002; Ure, 2015). In 
the Netherlands, jobseekers can gain access to 
RVA once their application has been approved 

by the national job agency (Duvekot, 2016). In 

the United Arab Emirates, a ‘Recognition of 

prior learning (RPL) charter’ guarantees mutual 
recognition of the outcomes of RVA processes by 
all registered providers (UAE, 2016). In all these 
cases, utilising the right also entails receiving 
guidance on compiling a portfolio of evidence for 
the assessment, as well as counselling on the 
outcomes of the assessment. 


These examples demonstrate the variety of 
opportunities available to newcomers in reaching 
out to the education system by utilising formal 

or informal rights to guidance and assessment 

in validation and learning processes. Greater 
accessibility, affordability and userfriendliness of 
RVA and the development opportunities it offers are 
important preconditions for opening up this right to 
all migrants and refugees. 


5.5.3 Assessment 

Assessment is crucial to RVA processes. The 
main methods of assessment are traditional 
examinations, assessments of portfolios of 
evidence, on-the-job observations and practical 
assessments. Generally, summative assessment 
is carried out, although formative assessment and 
reflective assessment can also strengthen the 
newcomer's inclusion. 


Formative RVA is used to inform and guide an 
individual who wants to embark on a course of 
study. It uses a newcomer’s learning biography 

to develop a personal learning path and design 

a tailored learning programme. Summative RVA, 
meanwhile, measures a learner's performance 
against a standard and can take two forms: RVA for 
a qualification or RVA for intake. 


e RVA for a qualification tests a learner's suitability 
for a qualification or diploma: an independent 
assessment procedure takes place before a 
learner is admitted to a school or university 
programme. 

e RVA for intake takes place after a learner enrols 
on a course or programme: an individual's 
prior learning achievements, whether acquired 


through paid or unpaid work, in-service training, 
continuing education, leisure activities, or 

basic education and training, are assessed 

as contributions towards the study course or 
programme. 


Our recommendation is that assessment methods 
should be used as a feedback tool and study 
progress instrument during the learning process, 
rather than just for summative purposes. Special 
attention must be given to reflective assessment 

by means of self-assessment. In general, such an 
assessment strengthens the ownership of learning 
by transferring responsibility from others (education 
partner, employer, etc.) to the learner. When 
learners understand how to channel their interest 
and curiosity, they gain the ability to motivate 
themselves intrinsically and to better understand the 
relationship between effort, strategy, persistence 
and use of resources in meeting their learning 
challenge: they gain the power to control what 

they learn. When learners begin to own their 
learning, they gain a prized possession that they can 
protect, build and maintain for a lifetime. In other 
words, they create their ‘learning independence’ 
(Rickabaugh, 2012; Duvekot, 2016). 


5.5.4 Professionalism and quality 

It is important for RVA staff (portfolio guides, 
advisers and assessors) to have a high level 

of expertise in order to generate and maintain 
awareness of the value of personal learning 
experiences when refugees and migrants seek 

to attain qualifications placed or available in the 
NOF. Furthermore, the quality assurance of an RVA 
system depends on the professionalism of the staff, 
how they acquired and maintain their expertise, 
and how they are embedded and accepted 

in qualification systems and human resource 
management systems on the labour market and in 
the third sector. 


The RVA process comes into its own and enables 
refugees and migrants to access quality education 
in their new countries of residence when (i) 
schools and universities acknowledge the value of 
prior personal learning experiences; and (ii) RVA 
staff members excel in the various RVA activities 
of outreach, guidance, portfolio management, 
assessment and counselling. 


5.5.5 Roles and responsibilities 

The responsibilities of stakeholders can be linked 
to the purposes of engaging in RVA. The various 
roles to be filled can be explained by pointing out 
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their respective tasks and responsibilities. Table 5.2 
shows the main objectives for each stakeholder next 
to their linkage with specific responsibilities. Many 
objectives can occur simultaneously in a national 

or sector context, since the true impact of RVA lies 
in the combination of certification, strengthening 

an individual's self-esteem and articulating the 
competence needs of learners. 


What is important is ensuring the impact of RVA 


at the individual level. This entails giving a voice to 
the learner, and is most effective if the learner is 


Table 5-2. Objectives and responsibilities of RVA 


fully aware of the social perspectives facilitated by 
the RVA process. Alongside good-quality guidance 
and assessment, creating perspective is a critical 
success factor for RVA in the education system. It 
is vital to be able to lead refugees and migrants not 
simply to achieving a qualification by means of their 
prior learning experiences, but also to finding a job 
and maintaining and improving their employability, 
enabling them to integrate into another culture and 
to empower themselves in developing self-esteem, 
to name just a few impacts and results from 
engaging in RVA. 


Refugees and e 
migrants ownership 


e Creating second chances 


Strengthening self-esteem and e 


Building a learning biography, portfolio 
or other type of document for recording 
personal learning experiences 


e Engaging in self-investment in e Agency: actively following ambitions 
learning and learning objectives 

e Gaining access to the education e Carrying out an analysis of strengths, 
system weaknesses, opportunities and threats 


e Taking advantage of social and 
economic career opportunities 


(SWOT) 
Compiling portfolio documentation 
Carrying out self-assessment 


Schools, e Accrediting prior learning e Open mind to validating prior learning 
universities and outcomes for further learning e Standards based on learning outcomes 
VET e Offering made-to-measure learning ¢ Facilitating lifelong learning strategies 
e Innovating education processes e Providing information on procedures 
e Engaging in society e Providing training for assessors, guides, 
counsellors 
e Conducting assessments (Summative/ 
formative) 
e Offering portfolio formats and portfolio 
training 


Organisations ° 


and companies competences 


e Formulating need for competences 


e Designing lifelong learning 


strategies in human resource 


Offering self-management of 


e Offering tailored learning pathways 


Providing mission statements 
Providing an inventory of the 
organisation's skills needs 

e Analysing the organisation's (company's) 
strengths, weaknesses, opportunities 


management and threats (SWOT) 
¢ Compiling competence-based 
occupational profiles 
Public e Linking to social opportunities e Offering self-management of 
employment e Newcomer reception services competences 
services e Guidance and advice e Mapping social opportunities (local, 
regional) 


(continued) 
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STAKEHOLDERS 


OBJECTIVES 


RESPONSIBILITIES 


National level e Establishing policies and e Facilitating RVA by legislation and 
responsibilities for creating funding, opening (learning) centres 
favourable conditions for lifelong e Raising awareness of the benefits of 
learning through laws, regulations RVA 
and funding e Enabling social and economic mobility 

e Promoting RVA for newcomers e Managing qualification structures or 


Sources: UIL, 2016; Duvekot and Duvekot, 2018. 


5.6 Conclusion 


RVA can be beneficial for all stakeholders involved 
both in qualification- or job-related approaches 

and in more self-directed pathways for learning 

and development. This perspective is supported in 
many countries around the world by legislation and 
national and sector policies on RVA. However, these 
policy initiatives have not yet been operationalised 
to their full potential in practice. 


At the system level, favourable conditions for 
creating learning and social perspectives are present 
in countries with (i) a legal framework for applying 
RVA in education and in the public and private 
sectors; (ii) an operationalised system for RVA in 
schools and universities that is able to recognise 
prior learning experiences of a formal, non-formal 
and informal nature; (ili) funding available at national, 
sector and/or NGO levels; and (iv) opportunities for 
anchoring RVA outcomes to competence-based 
systems in education, the labour market and the 
third sector. 


At the individual level, the main requirements are 
agency, information and guidance, and ownership of 
learning. Experience from different countries shows 
not only the slowness of policy implementation in 
practice, but also that an open dialogue is conducive 
to activating individual learners. In general, refugees 
and migrants are initially hesitant to access the — 
often rather complex — RVA processes, but once 
they have been informed and guided, and are 
reflective with regard to their learning experiences, 
they can become empowered and engaged. 


frameworks 
e Providing quality assurance of RVA 


Information provision on the potential benefits of 
RVA is crucial for the agency of the target groups. 
They feel valued whenever they are helped to 
demonstrate their prior learning and working 
experiences. They feel ‘listened to’ and are 
encouraged to engage in participative actions in 
their new country. Furthermore, once it has been 
built up, their portfolio can be regarded as the 
carrier of the RVA process. Guidance or mentoring 
in the documentation phase is clearly beneficial for 
refugees and migrants. 


Offering RVA as an open gateway for linking a 
newcomer's formally, informally and non-formally 
acquired learning achievements to a national 
qualification is one of the main critical success 
factors for their inclusion in their new country of 
residence. Following such assessment, the offer 

to acquire additional competences within a tailored 
and flexible learning path is another critical element 
for success. Both these factors depend heavily on 
the willingness and ability to recognise the skills and 
competences acquired outside the new country. 
This is not just a newcomers’ responsibility, but 
also a matter of ambassadorship and expertise on 
the part of the RVA staff, of open-mindedness and 
willingness to embrace a dialogue on validation and 
learning rather than just depending on assessment 
methodology or interview techniques. 


Finally, this analysis of RVA for refugees, migrants 
and displaced persons is consistent with Paulo 
Freire’s statement in the 1970s that learning needs 
to be addressed as a developmental and dialogical 
process in people's praxis by reflecting upon their 
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reality and so transforming it through further action 
and critical reflection. This should be an anti- 
depository process, contrasting with the traditional 
‘banking system’ (Freire, 1970). Such processes of 
dialogical validation and learning suit every learner, 
including refugees and migrants with their prior 
learning experiences. By ‘banking’, Freire means a 
process in which knowledge is directly transferred 
to learners with the teacher as the sole distributor of 
knowledge and the learner as the passive receiver 
of this knowledge. Instead of ‘banking’, a more 
suitable term to refer to the ground floor for learning 
might be ‘portfolio-ing’, in which learning is based 
on personal prior learning experiences and the 
self-management of recurring learning processes. 
Moreover, the role of the teacher can also be filled 
by a manager or team leader in the workplace. In 
this way, RVA adds value by making the learning 
process the object of learning, with the learner and 
the teacher (or manager) as ‘partners in learning’, 
openly debating the design and implementation of 
the learning that is needed or desired at the level of 
the refugee/migrant, teacher and manager. 


The authors would like to give credit to Madhu 
Singh, Senior Programme Specialist at the UNESCO 
Institute for Lifelong Learning, who died shortly 
before this chapter was published. Madhu’s 
research into RVA and its potential for improving the 
lives of refugees and migrants around the world will 
continue to contribute to the discourse on reaching 
some of the world’s most vulnerable people through 
education. 
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CHAPTER 6. 


ACCESSING AND ACQUIRING SKILLS AND 
QUALIFICATIONS: CONNECTING PEOPLE TO 
NATIONAL QUALIFICATIONS FRAMEWORKS 


Michael Graham, Arjen Deij, Mirjam de Jong, Jolien van Uden, Carmo Gomes and 
Eduarda Castel Branco, European Training Foundation 


Summary 


To deliver benefits to people — learners and 
employers — national qualifications systems need 

to be connected to system elements that allow 
those people to acquire skills and qualifications. This 
chapter considers how countries can enable people 
to access and acquire skills and qualifications, by 
looking at system elements such as curricula, career 
guidance, and teaching and learning. 


6.1 Skills, qualifications and their 
environment 


The ETF's previous contributions to the inventory, 
whether focusing specifically on national 
qualifications frameworks (NOFs) or qualifications, 
or more broadly on their supporting systems, 
were concerned with qualifications, frameworks 
and systems as a distinct thematic topic, or set 

of related sub-topics. Most ETF partner countries 
now have an NOF in place, and are modernising 
their qualification systems. But there is another 
stage before NOFs and qualification systems can 
connect to people to deliver skills and qualifications. 
That stage can be found in curricula, teaching 

and learning, and career guidance. Therefore, 

the question now is how countries’ reforms of 
their qualifications and systems, especially those 
grounded in the principle of learning outcomes, can 
facilitate people in knowing about and acquiring 
skills and qualifications. This can be observed 
principally by examining how they interact with the 
related areas of learning and teaching and career 
guidance. 


The ETF is conducting a new study on these 
themes, which we summarise for the inventory in 
this chapter. Here we seek to provide insights into 
how people access or know about, and acquire, 
skills and qualifications. We are guided by three core 
questions: How do people know about skills and 
qualifications? What types of skills and qualifications 
do they need? And how can people be supported to 
acquire them? 


According to the Europass Decision of 2018, ‘[...] 
skills are understood in a broad sense covering 
what a person knows, understands and can do. 
Skills refer to different types of learning outcomes, 
including knowledge and competences, as well as 
ability to apply knowledge and to use know-how in 
order to complete tasks and solve problems’. 


For qualifications we use the European 
Qualifications Framework (EQF) definition, which 
sees qualifications as the formal certification 

of learning outcomes achieved against given 
standards. 


We see a new balance arising between skills 

and qualifications. The debates and studies about 
requirements for the future demands of the labour 
market are all about skills and competences. 
Increasingly, skills are seen as the dynamic 
currency that people need in a fast-changing world, 
to which they must be able to adapt throughout 
their lives. This puts lifelong learning back on the 
political agenda; most countries, whatever their 
technological capabilities and economic status, are 
introducing policies to address the acquisition of 
‘218 century’ skills throughout people’s careers and 
lives. 


However, at a conference in November 2018, the 
ETF hosted one of the few discussions of the future 
of work and skills in recent years to concentrate 

on the experience of transition and developing 
economies. Entitled Skills for the Future: Managing 
Transition, the event drew on the ETF's broad 
European and international links and on its ability 

to examine issues from the perspective of its 
partner countries — a diverse and distinctive group of 
countries that share much geographical, historical, 
and cultural intimacy with Europe. Individuals, 
enterprises and education and training providers 

in these countries must make the best decisions 
they can about the skills and competences they 
need, and the qualifications they should pursue, for 
future success. In shaping their respective futures, 
three factors seem to have significant impact 
across ETF partner countries. First, each country's 
‘starting point’ consists of an analysis of where it 
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stands and where it wants to go. The second is 
making the best possible choices now to mitigate 
challenges and seize opportunities. And, therefore, 
the third is about institutional capacity, and relevant 
actors’ ability to work together. As the conference 
conclusions state (ETF 2019): ‘Shaping the future 
requires resilience, adaptability and agility from all 
actors’ 


Yet qualifications and qualification systems 

are seen as Static and unable to keep up with 
changes and to capture the wide range of new 
learning opportunities. New alternatives to formal 
qualifications are emerging, such as digital 
credentials and badges, to make the skills that 
people have acquired more visible. If we want 
qualifications to remain relevant for individuals and 
for society, we need to build more flexibility into 
qualifications and qualification systems. 


In part, the ETF’s new study, and this chapter as 

a summary, are prompted by a belief that, for the 
benefit of learners and employers (the main users 
of skills and qualifications), we need to look more 
often at the two together. We try to identify in what 
environments, and in what modes, people learn 
best; how they prefer to learn; and at what ages 
and stages in their lives and careers. We are also 
concerned with employers, particularly what types 
of employer may dip ‘below the radar’ of policy- 
makers and governments. 


When we refer to ‘people’, we are concerned 
primarily with the ‘learner’ and ‘employer’ 
categories. Throughout this chapter, it is these two 
broad groups we have in mind. 


The ETF works with 29 high-, middle- and low- 
income countries located around the European 
Union (EU) in South Eastern Europe, Eastern 
Europe, the Southern and Eastern Mediterranean 
and Central Asia. Given their great variety of 
economic and social conditions, and thus their 
various capacities to implement reforms, these 
countries are at different stages of qualification 
system reform. Linking policies and practices 
addressing the breadth of issues we are concerned 
about — from qualifications, skills development 
and career guidance to curricula and teaching and 
learning — is not easy. It may indeed be simpler 

to develop policies in related but distinct fields 

in parallel, without a clear strategy and a shared 
vision for education and labour market reforms, 
rather than attempt to tackle them in an integrated 
fashion that includes wider elements. But we 
observe that integrating policies and practices in 
skills and qualifications with their related system 
elements, such as guidance and curricula, applies in 


all countries at any stage of qualification system, or 
NOF, development. 


This chapter presents the findings so far from 

our study, including what we have discovered 

in examining how — by looking at the design of 
curricula, career guidance, and teaching and learning 
— countries can enable people to access and acquire 
skills and qualifications. Our findings are necessarily 
tentative, as the study Is a work in progress. 


We look at where people find information about 
skills and qualifications, and how they find it. 
Another motivation for addressing these issues now 
is a feeling that partner countries are moving only 
slowly in modernising their qualification and broader 
vocational education and training (VET) systems. 
Governments propose laws, establish institutions, 
create stakeholder platforms, design formal 
curricula, improve quality assurance tools, and so 
on. But reforms can remain unconnected to related 
or neighbouring policy areas, and if not properly 
implemented the effects in practice and benefits for 
people can be limited. 


6.2 Benefits for people 


The expert in anything was once a beginner. 
Helen Hayes, actor 


While it may seem obvious for an EU agency 
concerned with, among other issues, skills and 
qualifications to promote the worth of skills and 
qualifications, we think partner country decision- 
makers and policy-shapers need to make a case to 
their own societies. It is not axiomatic, after all, that 
education and training, skills and qualifications, are a 
priority in all partner countries, when examined in the 
context of all the other issues they have to deal with. 


Decision-makers need to understand the benefits of 
skills and qualifications, to individuals, employers and 
society. Many ETF partner countries are undertaking 
large-scale reforms, often aided by the EU and 
sometimes by individual Member States. VET 
continues to have a key role in preparing young people 
for the labour market, and, at the same time, needs 
to respond to the growing demand for upskilling and 
reskilling of adults at all ages and levels. By ‘adult’, 
we mean learners who have left initial education, and 
usually have some work experience. 


One of our study's findings relates to the effects or 
benefits of learning for individuals and employers. 
Learning with the goal of acquiring a set of skills or a 
qualification affords an obvious gain for the learner's 
employer, the broader economy, and, by extension, 
society as a whole. We know that those with skills and 
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qualifications are more likely to lead healthier, safer 
and longer lives, and are less likely to commit crime or 
fall into other types of anti-social behaviour. 


The ETF's remit is to support countries in developing 
systems for training people for work or employment. 
Yet our investigations reveal that governments and 
employers stand to gain by encouraging people to 
access learning and qualifications, even if the material 
benefit is not known at the outset. Surveys tell us 
that it is primarily the learning process that yields 

the gain to people in their personal and professional 
lives. Therefore, while acquisition of an occupational 
skill, which is then formalised in a qualification (i.e. an 
official attestation of outcomes achieved against given 
standards) is desirable, it is not the sole, or even the 
main, benefit to employers and governments. 


Intrinsic motivation is one of learning’s most 
precious resources. 

Rachel Simmons, author and women’s rights 
activist 


Learning among peers is frequently hugely satisfying 
for adults. Even without the target of acquiring a 
formal certificate, they gain social, digital, and other 
more generic skills that stand them in good stead in a 
range of occupations. This finding seems to be borne 
out by employer surveys, which generally accord 
higher priority to behavioural than technical skills. 
The BELL survey, which included Serbia (an ETF 
partner country), found that adult learners, especially 
those coming to courses with formally low skill or 
qualification levels, gain proportionately the most 
from training. Learning environment matters — we 
found that employees often do not like classroom 
learning, even if related to their job. 


It has been our experience at the ETF that adult 
VET or continuing VET (CVET) is more agile than 
secondary or initial VET (IVET), in responding to 
learner and labour market needs. Private providers, 
NGOs, donors and public employment services in 
some countries, for instance Kosovo’, adapt more 
readily to the lifelong learning paradigm, including 
providing training for occupational and social skills 
and outcomes-based qualifications. One of our 
findings is that governments need to give more 
attention to CVET and the non-formal providers, 
and, more broadly, to lifelong learning policies and 
practices. Governments and employers may not 
view adults as learners, especially when they are 
in work, yet the potential for a disproportionate 
economic dividend exists there. 


24 This designation is without prejudice to positions on 
status, and is in line with UNSCR 1244/1999 and the 
ICJ Opinion on the Kosovo declaration of independence, 
hereinafter ‘Kosovo’. 


At the same time, the positive effects of learning 

do not sufficiently reach young folk in our partner 
countries. The EU’s early leaver target is 10%, but 

in Turkey, according to studies supporting the ETF's 
Torino Process analysis, one in three 18-24-yearolds 
fall into this category, and in Albania the figure is 
40%. Numbers of young people not in employment, 
education or training (NEETs) are high in Bosnia and 
Herzegovina, Georgia and Kosovo. 


Typically, navigating the transition from school to 
further study, or early career paths, is difficult, and 
few young people in partner countries express 
satisfaction with the careers information or guidance 
they receive. VET graduates typically consult 
informal networks when job seeking, or respond 

to specific job advertisements rather than go via 
public employment services. In North Macedonia, 
only 11% of VET graduates used public employment 
services as a source for job searches. 


Employers form our second category of 
beneficiaries. Employees’ skills are a key factor 

at any time in business success. As the economy 
moves from industrial revolution 3.0 to 4.0, from 
computerisation and automation to cyberphysical 
systems, training and upskilling people matters even 
more. This transformation is especially challenging 
for smaller enterprises. Globally, small and medium- 
sized enterprises (SMEs) constitute upwards 

of 90% of all firms and more than 60% of total 
employment. They are more vulnerable to global 
changes, such as digitisation, and less productive?®. 


As with our findings about learners, or types of 
learners, we think countries may gain by giving 
special attention to SMEs. In the countries of South 
Eastern Europe that are currently candidates for 

EU membership, SMEs account for 90% of all 
firms, and provide about 70% of all jobs. Yet while 
smaller firms often lack the resources to upskill 
their people, larger companies are more likely to 
have their own training programmes and facilities. 
Furthermore, those with lower levels of training are 
disproportionately found in SMEs. By contrast, the 
better educated or more skilled tend to gravitate to 
bigger, private firms or to public institutions such as 
ministries and local authorities. 


SMEs' challenges begin with identifying what 
training is needed, and how to allocate their more 
limited resources. Broadly, they seek to upskill 
workers via informal training and learning in 

their own organisation. As indicated above, this 
learning — mentoring, staff meetings, company 
training sessions, etc. — often suits adults already 


25 See, for example, ICT (2015). 
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in employment. But in most cases in ETF partner 
countries, the employee's newly gained skills are 
not recognised through a qualification. It should 
be added, however, that older workers are less 
concerned with acquiring formal certificates than 
they are with updating their skills. 


One potential response is for industry bodies, 
chambers of commerce and other relevant actors 
to collaborate with training providers to develop 
more training courses and more flexible certification 
processes. 


6.3 Knowing about skills and 
qualifications 


Students need information, information, 
information. 
James Turner, head of policy, The Sutton Trust 


How people know about which skills and 
qualifications they might need is all about making 
information accessible. There is no shortage of 
information out there about skills and qualifications. 
Indeed, the internet provides a plethora of 
information, making it a challenge for individuals or 
employers to navigate through this sea of sources. 


Partner countries need to develop tools that can 
help employers and people sift through all this 
information, in order to find what they need to 
make decisions about skills and qualifications. 
Fragmentation and complexity, and thus 
inaccessibility of information, cannot be allowed to 
become a negative consequence of volume and 
speed. Instead, partner countries need to develop, 
or apply, tools to channel the flow of data to the 
benefit and not to the detriment of learners and 
employers. 


When it comes to knowing about skills and 
qualifications, we see three domains which 

are undergoing transformation: first, skills and 
qualifications systems and tools; second, labour 
market intelligence; and third, career guidance or 
counselling. Below, we explore each of these. 


6.3.1 Skills and qualifications systems 

Information about skills and qualifications serves 
several purposes, including supporting quality 
assurance, matching supply and demand in the 
labour market; informing career choices; and 
generating transparency, and ultimately trust, in the 
qualifications and skills held. 


The EU deploys multiple tools, notably Europass 
and the EOF, to generate access to, and present 


information on skills and qualifications. Europass 

is a portfolio of tools used directly by citizens, and 
is currently being overhauled by the European 
Commission and Member States to shift it towards 
becoming a comprehensive digital platform. 

The revised version will include information on 
demand and supply of jobs and skills, including 
through synergy with related tools such as the EU 
Skills Panorama, labour mobility tools such as the 
European Job Mobility Portal (EURES), the EQF and 
authentication measures to support verification of 
digital documentation. 


NOFs themselves help people to know about 
qualifications, because they are visible and, 
therefore, communicable entry points to a country’s 
qualifications. Indeed, helping people know about 
and understand qualifications is a goal of all NOFs, 
in our partner countries and universally among the 
150 or more in existence around the world. Having 
a register of qualifications that is accessible online 
is an important feature of many NOFs. National and 
regional qualifications frameworks have become 
integral to most countries’ qualification systems. 
They have not always been successful, though, 

in communicating their role and uses beyond 
stakeholders who are already active in the education 
and training world. 


The ETF's chapter in the 2017 inventory highlighted 
information and communication as a transversal 
function of the institutions, ministries and 
agencies that lead qualifications reform, or have 
lead responsibility for the NOF. So in the partner 
countries, it is essentially the agencies such 

as the Montenegro Qualifications Council, the 
Vocational Qualifications Authority in Turkey, the 
National Qualifications Authority in Kosovo, and 
other countries’ education and labour ministries, 
which lead in communicating about qualifications to 
learners and other users. 


These institutions use a range of tools to reach 
users. In Kosovo, the National Qualifications 
Authority uses films, adverts and websites to 
promote familiarity with the qualifications system. 
Among EU countries, Ireland has given special 
attention to communicating information on 
qualifications and related programmes. Other tools 
include registers and portals of qualifications. North 
Macedonia, Kosovo, Georgia, Moldova and Turkey 
are among several partner countries developing 
registers and portals of qualifications. Montenegro's 
register includes 160 VET qualifications at levels 3, 
4 and 5 of the NOF. Likewise, handbooks such as 
Georgia's and Kosovo's effectively communicate 
their qualification systems to learners and 
stakeholders such as employers. 
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Further transversal functions of institutions include 
quality assurance and regulation. Quality assurance 
systems and tools also generate information and 
communicate about qualifications. Indeed, this 

is a necessary function of quality assurance, as 

its primary purpose in qualification systems is 

to create trust among users. That trust relies on 
credible information communicated to learners 
and employers. In skills and qualifications, 

quality assurance is principally about qualification 
standards, assessment and certification. 


Standards are defined with labour market actors, 
such as employers and trades unions. Such 
governance processes both boost quality and 
relevance of qualifications — from use of labour 
market information to production of relevant 
qualifications via occupational standards, for 
example — and, by engaging employers themselves 
in qualifications development, ensure greater 
employer awareness. In Scotland, the Scottish 
Credit and Qualifications Framework staff and the 
Scottish Qualifications Authority engage with a 
range of non-formal providers, such as Microsoft 
and the Scottish Police College, to develop new 
qualifications, which have experienced rapid growth 
in recent years. These processes increase public 
contact with the country’s qualification system. 


In partner countries, we might cite the cases 

of Ukraine and Turkey, where the employers 

have been leading development of vocational or 
occupational qualifications based on occupational 
standards, thereby ensuring skills and qualifications 
development is not a state monopoly. 


Assessment methods and certification may also 
generate greater familiarity with qualifications. 
National exams and national certificates, overseen 
by national bodies, seek to ensure more consistent 
quality of outcome in awarding qualifications, 

as well as greater visibility via common formats 
and badging. In Azerbaijan, the State Student 
Admission Commission, and in Georgia the 
National Assessment and Examinations Centre, 
are both expanding their remits. NOF levels 
stamped on certificates also have an information 
value. Montenegro, North Macedonia, Turkey and 
Kosovo all make their qualifications more visible 
and comparable through NOF and/or EOF badging. 
As with standards and qualifications development, 
engagement in assessment by employers, e.g. 

in Turkey, also broadens public awareness of 
qualifications. Additionally, partner countries 
increasingly publish results and data on results, 
qualifications take-up, and so on. 


Quality assurance and information also intersect 
in the new fourth generation of qualifications 


frameworks, in which ways of assuring the quality 
of the new qualifications, such as digital credentials, 
are now needed. Partner country governments need 
to respond to digital and other modernisation trends 
with policies, systems and tools which identify, 
capture and recognise learning outcomes and, 
crucially, quality-assure them for credibility. 


6.3.2 Labour market intelligence 

Labour market intelligence (LMI) includes skills 
analysis and forecasts, which enable us to better 
understand trends and drivers in skills demand and 
supply and, conversely, to detect skills mismatches. 
These tools are essential sources for decision- 
makers in skills and qualifications. While partner 
countries are aware of this need, they struggle in 
some cases to disseminate intelligence to users. 


The EU Skills Panorama combines different existing 
datasets, such as those managed by the OECD, 
Eurostat, Eurofound and Cedefop, to provide a rich 
picture of the European labour market. Users are 
decision- or policy-makers and researchers more 
than learners. Features include a search engine 
based on the European classification of Skills, 
Competences, Occupations and Qualifications 
(ESCO), LMI guides and other toolkits. 


ESCO seeks to link education to work, and support 
skills anticipation and matching. It helps users by 
showing how jobs can be described in terms of 
knowledge, skills and competences; by specifying 
how such terms can describe the outcomes 

within a qualification; and by suggesting which 
qualifications EU countries consider relevant to 
which occupations. 


Labour market intelligence is being transformed by 
Big Data and new analytics. Big Data is a term used 
to refer to datasets that are too large or complex 
for traditional data-processing application software 
to deal with. The goal of utilising Big Data in LMI is 
to support decision-making, by adding value to, and 
going beyond, traditional LMI tools such as surveys. 
Big Data has notable advantages, in particular by 
reducing cost, speeding up distribution of data, 

and offering greater accuracy and accessibility. 
Through the growing digitisation of labour 

market and education and training processes and 
information, more sources that supply information 
on occupations, skills and qualifications become 
available. This includes large datasets on job 
vacancies, CVs, learning opportunities and curricula. 


Relevant data is collected by online tools such as 
EURES, Europass, the EU Skills Panorama and 
information sources such as online job boards, 
portals and public registers. These sources allow 
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refined, detailed, localised and real-time intelligence 
on skills and occupations, which could not be 
captured before the advent of Big Data. They map 
skills by occupation, to identify mismatches and 
obsolete skills, and to predict new occupations and 
skills. 


6.3.3 Career guidance 

People need to know what skills and qualifications 
they should develop and acquire in order to 

equip them to pursue their desired careers 

and occupations. Career guidance, or careers 
information, means any information and/or advice 
that supports people in making meaningful choices 
about learning and work opportunities, to which 
skills and qualifications are, of course, closely 
related. 


We referred above to the role of LMI. Reliable, 
objective and user-friendly LMI is essential in 
supporting individuals to navigate labour markets, 
and in supporting enterprises to find people with the 
right skills. Traditionally, governments have played 

a significant role in funding, collecting, organising, 
linking, systemising and distributing LMI. Often, 

LMI tools have been developed in the employment 
sector, but are not necessarily shared with career 
practitioners in the education and training sectors. 


LMI can be considered one of the weak features of 
national guidance systems. Cooperation between 
the key stakeholders could be promoted with a 
national mechanism in collecting, maintaining and 
disseminating reliable LMI of supply and demand in 
the labour market. Attention also needs to be paid 
to analysis of trends and predictions of future need. 
The representatives of different user groups and 
guidance professionals could provide a sounding 
board in developing a userfriendly and innovative 
LMI system and in increasing their capacity to use it 
critically in career learning. 


The use of LMI can be progressed by increasing 
individuals’ access to experiential and non- 
experiential forms of labour market information. 
More attention should be given to links between 
work-based learning and career guidance in 
validation and adult learning. 


In a sense, career guidance takes LMI and makes 
it useable by people. People need to be presented 
with LMI developed and channelled by public 
services but also need to go a stage further in 
using that information to make informed choices. 
Such LMI includes knowledge and tracking of skills 
trends, demands and gaps; skill requirements 

in occupations; transversal skills as they evolve; 


entry and progression routes through occupational 
families; and so on. 


The rapidly evolving workplace, or, more 

accurately, the new forms of employment and 

new occupations, are making adaptability, mobility 
and upskilling essential. Career pathways cease to 
conform to familiar, linear expectations. Instead, the 
number of transitions — where career guidance and 
career management skills are arguably most critical 
— is increasing. This increased career unpredictability 
implies greater career self-management by 

learners and workers. Accordingly, the EU's Council 
Recommendation from 2018 on key competences 
for lifelong learning includes a competence that 
covers career management. 


Rapid changes in the labour market mean that 
citizens need new skills to cope with various 
challenges throughout their lives, including more 
frequent career changes. Such career management 
skills describe what individuals need to navigate 
the labour market and to develop their own working 
ives. The development of career management skills 
contributes to workforce development, enterprise 
performance, and career and work progression 

for individuals. A national career competency 
ramework could be developed, with input from the 
education and employment sectors, and other key 
stakeholders. It should be linked in a meaningful 
way to other national skills or competence 
frameworks. 


£ 


In schools, career education programmes can be 
delivered as a stand-alone and timetabled subject, 
as a series of themes taught across different 
subjects in the curriculum, or through extra- 
curricular activities. A cross-curricular approach 
helps students to think through work-related 
issues in the context of different subject areas. 
Extra-curricular activities can include career weeks 
or career fairs, workplace visits, work experience 
programmes, or employer school visits. The most 
comprehensive approach is to include career 
education as a compulsory or optional timetabled 
subject within the curriculum. 


Career education programmes can be delivered 

by career teachers or career practitioners, based 
either in or outside the school. A third model is a 
partnership between school-based and external 
personnel. A well-functioning partnership model can 
link the curriculum content to a more up-to-date and 
in-depth knowledge of the world of work. 


Career education or career guidance is not one 
intervention, but many, and works most effectively 
when a range of interventions is combined and 
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sequenced in a programmatic fashion (Hooley, 
2014). Sultana (2018) distinguishes between the 
curriculum-based and work-based pedagogical 
methods in the delivery of career education 

in addition to career guidance services. The 
involvement of employers and working people 
in work-based experiential learning is critical for 
individuals’ career learning. 


For adults, guidance refers to a wide range of career 
counselling activities built on co-construction, by the 
counsellor and the client, of the client's relationship 
with working life, and on services and tools that 
help jobseekers to improve their employability 

skills and reintegrate in the labour market. This 
encompasses a wide range of activities, including 
assessment, profiling or screening, assistance 

in job changes and upskilling, coaching in career 
management and social skills, job-broking and 
advocacy, job-search assistance, counselling, 

job club programmes and provision of labour 
market information. For the unemployed people, 
public employment services’ counsellors provide 
counselling to support sustainable activation and 
labour market integration. Public employment 
services may offer specialist career guidance to 
those seeking a more comprehensive solution — for 
example, a low-skilled unemployed young person or 
adult who wishes to pursue a VET programme. 


Gaining and sustaining employer engagement is 
central to effective career development systems in 
addressing labour market gaps and mismatches. The 
cooperation structures depend on local conditions 
and needs and can range from information exchange 
to co-design and implementation of work-related 
learning activities. 


Work-based learning should be a core element of 
the education system for all, and should include 
learning about entrepreneurship and social 
enterprise. Employers can be engaged in national 
committees preparing national policy documents or 
legislation. They can contribute to the development 
of curricula for different educational levels and for 
training programmes of career practitioners, and 
provide placements for career practitioners, as 
well as students and teachers. Employers can also 
build on existing human resources management 
processes to create career development support 
for the mutual benefit of themselves and their 
staff, in cooperation with trade unions, professional 
associations and other work-related bodies. 


The professionalism and qualifications of career 
services providers are essential elements in 
meeting the needs of citizens and enterprises, as 
well as in furthering national policy goals. However, 


current training provision has been inconsistent, 
and reflects the nature of national guidance delivery 
systems. Insufficient attention has been paid to 
LMI and to competences in the use of technology 
and social media in guidance. Employers and other 
relevant stakeholders are not sufficiently consulted 
in the design and evaluation of career practitioner 
programmes. 


Guidance practitioner qualifications should be 
integrated with ministries or authorities running 
NOFs to improve the transparency of qualifications 
across countries. Citizens need to be enabled 

and assisted to identify their guidance needs 

and to have these addressed in a competent and 
professional way, so that they can acquire skills to 
make meaningful and efficient plans and choices for 
their learning and career. 


6.3.4 Use of ICT in guidance 

The relationships between career services 

and different user groups have become more 
dynamic and complex, with high demand for 

online customised products and services. The 

use of social media in careerrelated activities has 
increased dramatically in recent years, and the 
careers sector needs to expand its understanding 
of new technologies and labour market information 
systems (LMIS) as part of efforts to modernise 
services. It is necessary to exploit the potential of 
new technologies, alongside LMIS, and integrate 
their use into all aspects of lifelong career guidance 
practice and service delivery. The implementation of 
a strategic approach in the use of ICT and LMIS in 
relation to lifelong guidance requires a jointly agreed 
cross-ministerial strategy for lifelong guidance, and a 
common conceptual framework for service delivery 
and funding. Successful integration of technology 
requires coherent policy support, workforce capacity 
development, and the design of fit-forpurpose ICT 
systems. 


Given the diversity of career services in 

different settings, it is important that all relevant 
stakeholders participate in the development of 
systems and policies for lifelong guidance. It is 
equally important to share knowledge at national, 
regional and local levels. From a policy and delivery 
perspective, strategic leadership with stakeholder 
involvement across sectors helps to overcome 
policy fragmentation and duplication of efforts. 

It is essential to promote research to support 
evidence-based policy making, including market and 
academic research, longitudinal evaluation and cost- 
effectiveness studies. A coherent evidence base 
provides a rationale for allocating funding to different 
delivery channels and priority groups. Developing a 
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national strategy is critical to ensure that different 
sectors work effectively within complex systems. 


6.3.5 Some country cases 

Belarus and Serbia provide two innovative cases of 
outreach to users. In Belarus, the National Education 
Internet Portal supports decisions on programmes 
and qualifications though described learning 
pathways. Its users are learners, school pupils, 
university students, workers and adult learners, 
along with parents, school leaders and other 
stakeholders. Materials for curriculum development, 
counselling via email, and information on teachers’ 
continuing professional development, all feature. 


In Serbia, the Belgrade Open School Improvement 
of Employability programme aims to assist 
individuals in equipping themselves for current and 
future labour markets. As in Belarus, users include 
students, parents and other stakeholders. They can 
use a self-help tool to access information, or consult 
a career guidance practitioner online. Other features 
include self-assessment exercises, tools to create a 
career profile and CV, and even computer games to 
develop employability skills such as communication 
and decision-making. 


6.4 Acquiring skills and qualifications 


6.4.1 The skill of learning new skills 

Considering how people acquire skills and 
qualifications takes us into two broad areas, namely 
developing and implementing flexible curricula to 
provide opportunities for people to acquire skills 
and qualifications; and the changing role of teachers 
and trainers. In particular, we are concerned in this 
section with exploring innovative curricula, and 
identifying their consequences or implications for 
teachers. 


It will help to define what we mean by curricula 

and whom we are talking about when we come 

to teachers. A curriculum is a written plan for 
learning, which includes learning outcomes, learning 
goals, learning activities, and supporting materials. 
Curriculum development includes several stages 

of recurring cycles of design, implementation 

and evaluation. Evaluation criteria are relevance, 
consistency, practicality and effectiveness. 


Teachers instruct, supervise and assess learners, 
and may work in educational institutions, or in 
workplaces. They can be coaches, mentors, 
instructors, tutors, and so on. Here we examine 
how information about skills, standards and policies 
in qualifications can be translated into meaningful 
learning experiences. 


Learning outcomes can be achieved in different 
ways, but depend on appropriate inputs. Without an 
appropriate learning context, including competent 
teachers and opportunities for practice, it is not easy 
to achieve a system oriented to learning outcomes. 
We note that modularisation exists as a component 
in practically all EU VET reform projects in our 
partner countries. However, modularised curricula 
are not easy to implement, nor is modularisation 
part of the logic or experience of most existing 
systems. Nowadays, most students and teachers in 
our partner countries prefer a modular approach, but 
developing and maintaining one is hard work and is 
often not adequately supported with equipment or 
other resources. Critically, teachers are not always 
prepared for the task of incorporating complex 
reform measures into their daily work. 


We also see a trend for curricula to become broader 
and more general, covering key competences and 
sometimes reducing attention paid to specific 

skills and knowledge. This is in large part a result 

of the fact that much of what is learnt today can 

be outdated tomorrow. Flexibility and adaptability 
are increasingly among the measures of Success 

in the labour market today. The aim is for students, 
across all levels of education, to acquire transferable 
knowledge and skills more than they learn specific 
technical and professional skills. 


Inevitably, teachers are affected by these new 
approaches to assessment and teaching. Some 
distinctive features of outcomes-led education are 
listed below. 


e There is a focus on learning that combines 
knowledge and skills with personal and 
sociocultural competences. 

e Knowledge is set in context, and is 
interdisciplinary. 

e Relevant skill needs are central. 

e Learning is encouraged in a wide range of 
locations, and through different methods. 


Teachers are key to these changes, but system-wide 
reforms are not always system-deep enough to 
reach all teachers. Teachers need to be competent, 
but attitudes are important too if curriculum change 
is to be brought about. A major challenge for 
reforming qualification systems is, therefore, to 
choose an approach in which strategic objectives 
include the development of human resources. As 
this is often the most difficult and expensive part of 
reforms, it is often ignored. 


The ETF study Making Better Vocational 
Qualifications (2014) looked at what distinguishes 
a curriculum from qualifications — which is not 
always clear. We concluded that the purpose and 


GLOBAL INVENTORY OF REGIONAL AND NATIONAL QUALIFICATIONS FRAMEWORKS 


scope of curricula can be defined through the 

new qualifications, which can also provide a clear 
structure for the curriculum via use of modules. In 
most partner countries, vocational curricula are still 
highly centralised. In order to enable progression, 
many vocational curricula have been overloaded 
with general education subjects. There is a need 
to review this approach when redesigning curricula 
based on vocational qualifications. 


In Europe, there is a trend towards centrally- 

defined core curricula, with ample room for local 
implementation arrangements at the provider 

level. This also seems a good solution for partner 
countries. If the assessment is based on vocational 
qualifications, there is no need to micromanage the 
content of curricula. By giving providers the possibility 
to adapt curricula to the needs of specific learners 
and local companies, they can be more effective. 


Modular curricula have been difficult to implement 
in the partner countries, in particular when moving 
beyond pilot projects. Often, capacity building 
with teachers and trainers has been insufficient 

to introduce new curricula beyond pilot schools. 
Teachers also need time to train and to contribute 
in professional discussion. Feedback from countries 
that have moved towards modular curricula shows 
that the teaching of modules has already become 
routine in many schools, but the assessment of 
modules is still problematic. 


Developing learning outcomes is not simply a 


process of using the competences identified in the 
occupational standards; it requires interpretation to 


Table 6-1. Key institutions 


DESIGN AND 
IMPLEMENTATION 
lO) Com nO) ES 


INSTITUTIONAL ROLES’ 
TYPICAL EXECUTORS 


come to the statements about knowledge, skills and 
wider competences. 


In Qualification Systems: Getting Organised (ETF, 
2016), we looked at the regulations and the roles 

of stakeholders, institutions and quality assurance 
arrangements. We saw that education ministries 
had been focusing on curriculum reform, and in 
particular widening existing programmes. However, 
labour ministries have tried more to ensure that 
occupational descriptors reflect changing labour 
market needs, and have focused on labour market 
outcomes. It is often the labour ministries that start 
to work with employers’ representatives or social 
partners on training programmes for jobseekers, and 
on certificating adult learning (including non-formal 
and informal). It is therefore important that reforms 
involve different stakeholders and institutions from 
beyond the education system. 


Appropriate resources and capacities are required 
to translate qualifications into training programmes, 
and organise an appropriate learning experience 
focused on an active role for learners, including 
exploring alternative options. The learning 
experience requires curriculum development 
activities, teacher preparation and retraining, 

and the availability of learning tools and learning 
environments. Standards need to be fit for purpose 
to facilitate the improvement of VET curricula 

in terms of labour market needs and career 
development. 


Teachers and trainers are important, because they 
shape learning. Table 6.1 maps some of the key 


EXAMPLES 
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institutions involved in modernising the contents of 
vocational education. 


6.4.2 Designing curricula 

Designing and developing curricula are complex 
processes. Given the current context of rapid 
change, it becomes even more challenging not 
only to develop up-to-date curricula, but also to 
keep them up-to-date. Therefore, curricula should 
allow for flexibility to adapt the curriculum based on 
requests for new knowledge and skills. A flexible 
or semi-permeable curriculum often consists of a 
fixed backbone of fundamental content that does 
not change quickly, and a more open part that can 
be further designed and developed by institutes and 
teachers. 


Before explaining more about the curriculum design 
process, it is important to be aware of the different 
levels at which curricula can be developed, and the 
different representations of curricula, as this might 
influence curriculum decisions and will be important 
for curriculum evaluations. 


Curricula levels 
In general, the following four levels are 
distinguished: 


1. macro — often referred to as system or national 
level, 

2. meso -—or school/institute level, 

3. micro — the level of the classroom, and 

4. nano — the level of the individual student. 


For flexibility of curricula, it is important to notice 
the degree to which curricula are fixed at the 
macro and meso levels. In other words: How far 
can schools or institutes decide on the content of 
a curriculum? More open-ended curricula allow for 
more adaptation at school level, and make it easier 


Table 6-2. Representations of a curriculum 


INTENDED 
CURRICULUM 


IMPLEMENTED 
CURRICULUM 


Perceived 


Operational 


ATTAINED 
CURRICULUM 


to react to changes in the regional context and to 
respond to demand in the labour market. 


Another aspect to consider before starting to 
design and develop a curriculum is the different 
curriculum representations. With each step, 
interpretations are made by the users, and those 
interpretations can differ from the intended 
curriculum and, therefore, can influence the final 
results. These representations are extremely useful 
when monitoring and evaluating the processes 

and outcomes of a curriculum design or re-design 
process. 


Table 6.2 shows the different representations of a 
curriculum (Goodlad, 1979; van den Akker, 2003). 


Each curriculum starts with ideas on what needs 

to be accomplished, and the best way to reach 

the desired results. Especially at the higher levels, 
more than one person will be involved in the 
design process. The written curriculum is therefore 
the result of discussions between the different 
stakeholders. Hence, it will be a compromise of the 
individual ideal curriculum each stakeholder had in 
mind. Once there is a formal or written curriculum, it 
needs to be implemented. Teachers, who will often 
be the key actors in that implementation, will have 
their own interpretation of the written curriculum, 
and based on that they will plan and deliver their 
classes. In the interpretation and implementation, 
changes can occur that cause the curriculum to 
differ from the initial vision and ideas. 


The implemented curriculum will result in an 
attained curriculum. The attained curriculum consists 
of the learning experiences of the learners and 

the final learning outcomes they achieved. This 
attained curriculum also plays an important role in 
curriculum evaluation. Did the learners acquire the 
skills, knowledge and attitudes the curriculum aimed 
for? And if not, how can differences be explained? 


Curriculum as interpreted by its users 


Actual process of teaching and learning 
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Is it because parts of the curriculum got lost in 
the translation processes between the different 
representations, or because of design decisions in 
the curriculum itself? 


Taking into account both the curriculum 
development levels and the representations, we 
can conclude that the written curriculum can take 
different forms at the different levels. In the case 
of very detailed national curricula, the written 
curriculum at macro level will be the same as on the 
micro level. Based on the trend towards centrally- 
defined core curricula, with ample room for local 
implementation arrangements at the institute level, 
the written curriculum becomes more detailed at 
each level. 


Occupational standards describe the 
professional tasks and activities, as well as the 
required competences, for a specific occupation. 
Occupational standards answer the following 
question: What should someone in this specific 
occupation be able to do when employed? 


Educational standards define the expected 
outcomes of the learning process, leading 

to the award of a qualification. Educational 
standards answer the following question: What 
does the learner need to learn to be effective in 
employment? 


Assessment standards specify the assessment 
object, performance criteria, assessment 
methods, and who will be entitled to award the 
qualification. Assessment standards answer 

the following question: How will we know that 
the learner has acquired all competences and 
knowledge required to start working in the given 
occupational field? (Mansfield and Schmidt, 2001) 


Macro and meso level 

At macro level, the up-to-date occupational profile 
will be the primary source of input for the start 

of the curriculum design process. Based on this 
profile, educational and assessment standards can 
be defined. To allow different pathways (via formal, 
non-formal and informal learning) to lead to the 
same qualification, assessment standards will have 
to be formulated independently of the programme. 
It should not matter which pathway the candidate 
has followed as long as the candidate is able to 
demonstrate that s/he has acquired the required 
skills, knowledge and attitudes. It is important that 
assessment standards be formulated in such a 
way that they lead to an objective, valid and reliable 
assessment process. 


The outcome of the curriculum design process at 
national (macro), and regional and/or school level is a 
written curriculum that guides teachers in preparing 
their daily classes. In some countries a detailed 
curriculum will be designed and developed at 

macro level, while in other countries a more general 
curriculum framework is developed that will be used 
at meso level to design the final curriculum. 


One of the first decisions to make in the curriculum 
reform process is how detailed the curriculum 
needs to be at macro level. With a more detailed 
curriculum at macro level, all schools or institutes 
will implement a more or less similar curriculum. 
This might be easier to control, but it provides less 
flexibility. With a more open-ended curriculum at 
macro level, schools, institutes and teachers have 
more freedom to adapt the curriculum based on 
developments in their region. 


The key word here is flexibility. Whereas, 
traditionally, curricula were micromanaged at central 
level, the trend is to leave more and more space for 
teachers, trainers and learners to shape the learning 
experience in order to adapt to local contexts and 
individual needs. Where subjects, content and 
contact hours were dominant in traditional curricula, 
flexible curricula are modular in structure and 

can be organised in different ways. The modules 
define the expected learning outcomes, and 

are often interdisciplinary in content, integrating 
theory and practice rather than segregating skills 
and knowledge, as in traditional curricula. As the 
modules define what results are expected, rather 
than how they are to be achieved, it allows for using 
different approaches. 


Designing the written curriculum can start with 

a curriculum conference. The conference could 
define trends, explore and translate trends into 
the curriculum, and define learning outcomes, 
design principles and pedagogical starting points. 
The conference should produce scenarios offering 
different options, and create a network for further 
development. Stakeholders should participate 

in this conference to ensure relevant outcomes. 
Participants could be national representatives of the 
specific occupation, educational methodologists, 
teacher representatives and, preferably, student 
representatives as well. 


Issues that need to be addressed during the 
conference include: 


e Who will be implementing the curriculum 
(vocational schools, private training providers, 
etc.) and what does this mean for the level of 
detail in the written curriculum? 


Volume |: Thematic chapters 


81 


82 


e Besides the occupational standards, what other 
competences and skills need to be addressed in 
the educational standards and which ones would 
subsequently be part of the curriculum? 

e Where the answer to the previous question 
results in an overloaded curriculum, the follow-up 
issue is to rank the competences and skills from 
most to least important. 

e What is the role of different learning contexts, 
and to what extent do they need to be described 
at the macro level (e.g. the position of workplace 
learning)? 


The outcomes of the conference should include 

a first draft of the educational standards. These 
standards are the primary source for the curriculum 
design process. Where schools have considerable 
autonomy in designing the curriculum, the 
educational standards can be sufficient, and schools 
will use them to develop the curriculum for the 
programmes they provide. In contexts with a 
stronger national curriculum, working groups could 
be established to develop the curriculum based on 
the educational standards. The second stage of the 
curriculum design process will focus on the actual 
curriculum: What kind of programme is needed to 
meet the educational standards? 


Micro level 

The actual models can be designed at micro level, 
by teachers teaching the same modules. They will 
have to plan the activities needed to achieve the 
learning objectives of the module by the end of the 
module. This is similar to the process described 

in creating the curriculum overview, but at a more 
detailed level. It is recommended that students be 
involved in this process, as they can provide more 
insight to their preferred learning activities and 
expectations. 


The design process 

Designing or revising a curriculum is a complex 
process. The desired outcome of this process is an 
up-to-date aligned curriculum. Aligned means that 
all activities and entities of the curriculum contribute 
to achieving the defined learning outcomes. They 
are the best way to facilitate the learning process. 

It also means that there are no gaps or endless 
repetitions in the programme and that the different 
activities build on each other and are provided in the 
right order. 


For example, learning how to ride a bicycle can help 
to illustrate the meaning of alignment. The learning 
objective could be the ability to ride a bicycle 

from A to B ina safe way and without falling off. A 
programme consisting only of theoretical classes 


will probably not be enough to prepare someone to 
achieve this, even if it includes theory on balancing 
a bicycle. It is unlikely that anyone would be able to 
ride a bicycle based on only theoretical classes. This 
means the programme is not suited to achieving 
the learning objective, and therefore is not aligned. 
To be able to ride a bicycle in a safe way from A 

to B, theory about, say, traffic rules needs to be 
combined with practical sessions that give the 
learner real experience of riding a bicycle. In the 
optimal scenario, learners will have guided sessions 
in authentic traffic situations. Theoretical classes on 
how to balance a bicycle are probably not needed. A 
programme that takes into account these different 
aspects is purposefully designed to facilitate 
learning and, therefore, more aligned. 


A curriculum design process is often based on 
‘backwards’ design. This means that the final 
learning outcomes, and the way they will be 
assessed, form the starting point of the curriculum 
design process. 


6.4.3 Innovative curricula and pedagogies 

In line with the foregoing discussion, and based on 
our current understanding of how learning takes 
place, innovative pedagogies are being developed 
and implemented. The pedagogical landscape 
offers a wide range of options to choose from. This 
diversity can be overwhelming, and the best advice 
is to evaluate the options with the design problem 
at hand. As the educational context will differ by 
country, region and programme, we will describe 
some of the latest developments in pedagogical 
approaches. 


In general, during the last few decades it has 

been argued that activating learners in the 

learning process so that they construct conceptual 
knowledge and understanding themselves is more 
effective than lecturing and instructing alone. In 
instructional science, three main instructional 
theories are generally discerned: behaviourism, 
cognitivism and constructivism (Ertmer and Newby, 
1993; 2013). Each of these learning theories has its 
own answers to questions about effective learning 
and instruction, such as: How does learning occur? 
Which factors influence learning? How should 
instruction be structured? 


To deliver innovative curricula requires varied 
pedagogies. We have identified three learning styles 
that stand out: experiential, dialogic and blended. 

In essence, they hold much in common; a practice 
orientation, cooperation and dialogue between 
teacher and learner, and openness to diverse 
sources and methods of information, knowledge 
and expertise, and place of learning. 
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e Experiential learning is translatable as learning 
by doing. Learning environments should 
create opportunities to gain new experiences, 
promoting action and reflection to encourage 
reflective practitioners, including reflection in 
action, i.e. thinking while executing a task. 

e Dialogic learning suggests a partnership of 
two people communicating to learn together. 
Learners explore and share knowledge, using 
reasoning techniques, questions and sentence 
starters. Collaborative learning expands the 
group numbers. 

e Blended learning combines face-to-face learning 
with online or digital learning. It can include 
collaborative learning, or forums. Classroom 
learning might focus on content, while digital 
learning focuses, broadly, on skills development. 


6.4.4 Teachers and trainers - experts, designers 
Experience shows that top-down educational 
reform is a recipe for failure. The aim of professional 
development is, first, to change a professional's 
mindset from restricted (relying heavily on existing 
experience to face new situations) to extended 
(adopting a rational approach to new situations, 
supported by theoretical underpinnings of practices 
and experiences). This requires explanations, and 
the possibility to make further improvements if 
necessary. The focus needs to be on what the 
educational reforms are about and how they could 
be implemented, rather than on why and when 
(which tend to provoke attitudinal issues that all 
teachers have). Explaining the reform is a first 
necessary step for teachers to understand how their 
own practices will be affected. 


Teachers will only be able to enter functional training 
if they perceive the need for change in their own 
practices. Teachers must have the space to relate 
reform elements to their own ideas. 


End users (teachers and learners) need to take part 
in design processes. Teachers need to become able 
to design and re-design curricula, materials and 
learning environments, and align these with new 
assessment techniques. Curriculum development 
takes place in steps, typically including: 


e needs analysis, 

e defining objectives and outcomes, 

e developing a prototype, 

e implementing and evaluating, 

e redesigning, and 

e further implementation and dissemination. 


Design-based research introduces evaluation 

of relevance, consistency, practicality, and 
effectiveness during all phases. There is a strong 
focus on the validity of contexts. Most teachers 


have some experience with designing educational 
materials. Teachers often face closed problems 

in which the initial situation and outcomes are 
known, but these can become major problems if 
there is not enough support available (for instance, 
pedagogical content and process are poor, there is 
no supporting material, or there is a lack of teacher 
expertise). Therefore, teachers will need to learn to 
become designers in steps, moving from simple to 
more complex tasks. 


Design is an iterative process. Collaboration 
between teachers and stakeholders from outside 
the world of education and training is very important 
at the beginning of the process. The design process 
should become as participatory as possible and 
include end users. Multiple voices are key to 
successful adoption, redesign and implementation 
of an intervention. Therefore, the design process 
should be organised as a collaborative effort of a 
team of teachers. Curriculum development at micro, 
mesa and macro levels is strongly interconnected. 


VET curricula should be co-created, and connect 
teachers and workplaces in regional networks. 
This can be facilitated by school leaders, as 

well as by external facilitators. They can provide 
logistical support, and scaffold and monitor the 
design. During implementation there has to be 
the possibility for re-design (design after design), 
for example, by using an implementation scenario 
that invites teachers to analyse their understanding 
of the design, record local decisions, and collect 
anecdotal evidence on implementation. 


Teachers can be invited to adapt design products 

to their own situation and thus ensure adoption 

and implementation. Teachers need to reflect on 
the proposed innovations, and align them with 
personal capacities and practice. In order to create a 
conducive environment for implementing innovative 
curricula, teachers need to be obliged to actively 
relate new materials by selecting and interpreting 
them, and reconciling them with their own and 
their students’ beliefs. Design products that actively 
support redesign by local teams of teachers consist 
of building blocks and reusable resources, and 

are easily accessible. In addition, the creation of 
principle-based scenarios, which describe global 
lesson plans embedded in a strong pedagogy, 
leaving space for adding detail, is seen as helpful. 


6.5 Conclusions 
Qualification systems can look like perfect 
constructs that are difficult to implement. That is 


why many people prefer the language of ‘skills’ 
rather than that of ‘qualifications’. In order to move 
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from paper to practice, countries should address 
qualification system reform integrally with career 
guidance, information needs, curricula and teaching. 
This is of great importance to ensure that individuals 
and employers can obtain benefits from skills and 
qualifications. 


Learning as a process or experience yields gains 

to learners and employers, even where there is 

no formal certification. Adult learners already in 
employment benefit most from on-the-job training 
and other informal settings, compared to formal 
environments. Older learners, in adult education and 
CVET, are undersupported by government, but are 
especially receptive to learning opportunities. When 
supporting employers, partner country governments 
should prioritise the skills and training needs of 
SMEs, who in many cases lack the means to train 
staff independently. 


The modern era is one of information overload. 
Where do you start when you want to know what 
kind of skills and qualifications you need? There is 
plenty of information about skills and qualifications. 
Qualifications, labour market intelligence and 
careers management are three areas challenged 
and changed by modern data flows. NOFs, 

labour market intelligence and career guidance all 
facilitate knowledge of skills and qualifications. 
NOFs are underused as tools to communicate 
about qualifications. For the individual, career 
management is itself a vital skill now. At societal 
level, moreover, career guidance is arguably 
changing at a more rapid pace than some of the 
other systems and tools discussed in this chapter, 
because it is so closely linked to information 
sources, it is ICT and web-driven to a greater 
extent than the other areas we are discussing, and 
because individuals drive it in ways they cannot do 
in terms of directly influencing developments in 
curricula or qualifications. 


In the development and modernisation of curricula 
and qualifications, partner countries can struggle 

to manage the transition to learning outcomes. 
Decision-makers should step back from over 
prescribing what learners have to learn, and allow 
teachers and trainers to develop new curricula that 
promote the three broad categories of learning 
identified above — namely experiential, dialogic and 
blended - which seem especially well suited to VET. 


The role of teachers and trainers is changing and, if 
anything, expanding. Partner countries need their 
expertise in multiple, boundary-crossing roles, and 


in particular in functioning as curriculum designers. 
The strict separation of roles between classroom 
teachers and practice masters in schools and 
trainers in companies should diminish, in order 

to integrate theory and practice and strengthen 
learners’ problem-solving capacities. In practice, 
teachers are acquiring more varied roles than that of 
instructor. They are experts, coaches, and curriculum 
designers and developers, working in teams to co- 
create VET curricula. Everybody acknowledges that 
more forms of learning exist now — from self-study 
and workplace-based to online and digital — granting 
learners more choice and altering the learnerteacher 
relationship. But this does not constitute a paradigm 
shift from teaching to learning. 


Instead, attention needs to be given to the changing 
nature of that relationship and, indeed, one should 
consider if the role of teachers becomes more 
necessary in aiding learners to structure their 
learning and make sense of the evershifting world 
around them. 


People sometimes see qualification systems 

as being detached from education and training 
systems, focusing only on setting standards and 
separating assessment and certification from 
education. However, benefits for learners largely 
depend on connecting education and training with 
other parts of qualification systems. 
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Conclusions 


In contrast to a focus on the creation and early 
establishment of qualifications frameworks in the 
conclusions of the third edition of the inventory, 
the present edition discusses trends that relate 

to a wider set of issues. This reflects both a 
challenge and an opportunity for regional and 
national qualifications frameworks. Their impact 

iS growing in line with longer and increasingly 
effective implementation, but that inevitably brings 
them closer to areas that are undergoing significant 
change in policy and practice, or are facing the 
consequences of global factors beyond the scope of 
education and training systems to mitigate. 


1. Migration and mobility continue to spur efforts 
to prepare education and training providers, 
employers, authorities and migrants themselves 
to foster skills and promote employability. The 
internationalised labour market contributes 
to a growing need for comparisons between 
different systems’ outputs, so that those 
outputs remain fair and sustainable. At the 
same time, the greater mobility of education 
institutions, through overseas sites and study 
programmes as well as ‘borderless’ digital 
opportunities, contributes to new types of 
credentials. 

2. The World Reference Levels (WRL), constitute 
a response by the international community 
of qualifications experts to the growing 
internationalisation of labour markets, learning 
across borders, and migration and mobility. A 
group of experts, who collectively represent 
nearly all the world’s countries, is currently 
developing and testing a tool to describe, 
compare and recognise people’s skills, using 
learning outcomes as the conceptual basis, and 
so promoting a shared language of description, 
comparison and understanding. This has already 
proved a useful tool. For example, the Maldives, 
a member of the South Asian Association 
for Regional Cooperation (SAARC), is already 
working with other member countries to design 
and develop mechanisms for recognition, 
validation and accreditation of tertiary education 
qualifications. 

3. In turn, the WRL both respond to, and draw 
on, the digital technologies which increasingly 
influence the development of qualifications. 
There is no longer any doubt about the 
potentially transformative impact of digital 
technology on education and training systems; 
the question is how that transformation 
can support inclusive and equitable quality 


education, and the promotion of lifelong learning 
opportunities for all. 

Developments in micro and other alternative 
credentials are gaining pace around the world. 
Responses in Europe include, for instance, the 
Digital Education Action Plan, which supports 
the spread of digital competences and calls 

for a framework to recognise and validate 
digital credentials. In the US, an attempt at a 
comprehensive database capturing everything 
from open badges to PhDs is already in its 
third year of operation. The current review 

of the Australian Qualifications Framework 
(AQF) is also considering the inclusion of 
micro-credentials, or other forms of alternative 
credentials including from MOOCs, into the 
framework. These credentials may meet some 
of the criteria for inclusion in the AOF, although 
one important test is meeting government- 
approved standards. 

We see a co-existence of ‘traditional’ 
qualifications with newer credentialing methods. 
In fact, the need for an individual to be equipped 
with formal certification to enter the labour 
market is growing, e.g. in Europe. However, 

it remains the case that in many economies, 
especially in transition and developing 
economies, people find work without formal 
certification. The open issue is the relationship 
between traditional, formal certification and 
digital credentials. Quality assurance — the set 
of systems, tools and methods, which generate 
the all-important ingredient of trust — remains 
essential, and countries and international bodies 
need urgently to develop solutions. 

The rise of outcomes-focused approaches 
across post-compulsory education and 

training settings is now one of the most 
significant trends worldwide, both in vocational 
education and training, and, though in some 
countries to a lesser degree, higher education. 
Comprehensive frameworks including all levels 
and types of qualifications are gaining ground 
in Europe and elsewhere. European countries 
use their frameworks to create comprehensive 
maps of qualifications, including all sectors 
(VET, higher education, general education, adult 
training) and to help validate non-formally and 
informally acquired competences. This is seen 
as central to policies fostering people's lifelong 
learning and progression through different 
pathways. 

Countries such as Azerbaijan and Ukraine 
establish in law outcomes as the basis of 


qualifications and correspondingly stimulate 
development of associated tools such as 
occupational standards. At the same time, 
countries including Georgia and Moldova are 
moving to integrate qualifications frameworks 
for VET and higher education, hitherto only 
weakly linked. In Malaysia, the NOF was 
recently revised into a comprehensive tool that 
includes outcome-based qualifications in TVET 
and higher education. So, the learning outcomes 
principle is embedded in both policy and 
practice, albeit implementation may sometimes 
lag behind policy. 

The rise of validation of non-formal and 

informal learning (VNFIL), also called, variously, 
recognition of prior learning or RPL, and 
recognition, validation and accreditation of prior 
learning, or RVA, is making learning outcomes 
visible and valued, as part of efforts to reduce 
or remove barriers between learning and work 
and, therefore, aid career progression. Learning- 
outcomes-based qualifications frameworks have 
been set up relatively rapidly in some countries 
to support learners, providers and employers. 

In some cases, progress may be more on paper 
than real, but in most countries progress is 
visible in implementation. Those that have the 
firmest foundations increasingly develop linked 
systems of quality assurance. 

Outcomes also have a management dimension. 
They share the output focus of established 
management theories such as management 

by objectives and new public management, 
which, like outcomes approaches in education 
and training, combine central steering with 
local autonomy and scope for adaptation, 

in education, meaning regionally, locally, by 
schools and by individual teachers. 

Although evidence is necessarily limited by 

the recent application of learning outcomes, 
there are early indicators, which suggest the 
extent of its impact. In Europe, outcomes 

have improved visibility and transparency, and 
so understanding, of qualifications, through 
creating a common language. They also act as a 
tool to identify gaps in provision, for example in 
higher VET levels, as in Estonia; and to improve 
equivalence between types of qualifications. 
This is in part due to connections between 

the launch of the European Qualifications 
Framework, particularly — but not only — in 
Europe, the use of learning outcomes, and the 
spread of national qualifications frameworks as 
‘platforms for cooperation’. In Eastern Europe, 
progress in NOF development is most visible 

in increased numbers of outcomes-based 
qualifications, the adoption of tools such as 
registers and the advance of VNFIL. Elsewhere, 


10. 


11. 


12. 


13. 


Turkey and Serbia are now making use of 
extensive qualifications registers and databases. 
Studies are beginning to reveal the impact of 
learning outcomes on the management of 
education and training, but there are gaps in 
the process of moving from learning outcomes 
plans to teaching practices. Similarly, despite 
the apparent incorporation into pedagogies 

of the learnercentred perspective linked to 
learning outcomes, teaching practices seem not 
to be influenced to a comparable degree. More 
needs to be done to explore and explain the 
benefits, as well as the challenges, of learning 
outcomes, and to understand the real-world 
experiences of teachers and trainers tasked 
with applying them. 

Outcomes also promote changes in 
assessment. Learner attainment of intended 
outcomes is measured against common 
standards and criteria. Assessment under 
outcomes-based approaches is criterion- 
referenced, moving systems away from 
comparative or norm-referenced assessment. 
In principle, criterion-referenced assessment 
should result in greater equity and fairness, as 
the objective is to assist all learners to achieve 
the intended outcomes. 

Although it can be complex to implement and 
hard to scale, VNFIL is attractive to policy- 
makers and national authorities as it can reduce 
qualifications deficit among adults, and lower 
the costs of re-skilling and re-qualifying less- 
qualified people. India has ambitious plans for 
skills recognition and certification for people 
employed in the informal sector. VNFIL will 
here contribute to formalising labour markets. 
VNFIL can also be a tool to recognise the skills 
of migrants and refugees, thus supporting their 
economic and social inclusion. 

However, implementation, or wider application, 
of VNFIL is hindered by difficulties in gathering 
data to inform more effective policies and 
practices. VNFIL concerns very individual, and 
voluntary, learning pathways, so lack of data is 
perhaps an inevitable corollary. The integration 
of data systems to take account of VNFIL and 
the use of digital technologies to develop and 
maintain learners’ records — an existing trend 
as well as an explicit intention in places — 
presents important opportunities for better 
systemic analysis. The National Learners Record 
Database, operational in South Africa since 
1999, is an example of such a data management 
system. It collects and manages information on 
most aspects of the South African education 
and training system, including qualifications 
and part qualifications registered on the NOF; 
recognised professional bodies registered on 
the NOF; accredited providers and registered 
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14. 


19% 


16. 


WH 


assessors; and learners’ records detailing 
achievements of NOFregistered qualifications. 
One important issue in developing and using 
electronic databases is data confidentiality, 
which must be handled with due care and 
attention to individual privacy rights. 

A longitudinal study conducted in France points 
to high levels of satisfaction among those 
accessing the country’s VNFIL processes, 
including an encouraging proportion who report 
concrete personal and career improvements. 

In all countries, studies suggest a positive 
impact from VNFIL on people's experiences and 
outcomes, so vindicating the additional efforts 
to develop and adapt information systems to 
record evidence. 

To make VNFIL initiatives viable for migrants 
and refugees, they need support and guidance 
to be able to present their needs, as well 

as their abilities and prior learning, in a way 
that links with the needs of the society to 
which they have moved. This is all the more 
necessary for women because of systemic 
gender bias in both sending and receiving 
countries, as well as for individuals with special 
educational or other needs. Some countries 
offer assessment services, which aim to 
match individuals with potential opportunities. 
Countries are combining human rights and 
labour market considerations to both capture 
new arrivals’ skills, and allow for their personal 
and vocational development. 

As much as individual learners need to be 
flexible enough to be able to seize opportunities, 
so too systems need to offer flexible pathways 
in order to allow newcomers — and, indeed, all 
learners — to succeed. To make that flexibility 
work requires the recognition of prior learning 
and appropriate supporting mechanisms. Many 
countries acknowledge the need to reform 
VNFIL from legal, financial, labour market, 

and civil society perspectives, and to align 
VNFIL with the implementation of national 
qualifications frameworks. In the Republic 

of Korea, for example, the Lifelong Learning 
Account System is designed to validate all types 
of learning by promoting the recognition and 
certification of both academic and vocational 
qualifications. This allows individuals to 
accumulate lifelong learning experiences, and 
to use the qualifications obtained in the labour 
market. 

While much good practice is in evidence, more 
needs to be done to realise the full potential, 
both in terms of newcomers’ qualifications and 
employment, and in their personal learning and 
development. Integrated social, economic, and 
educational perspectives offer the best setting 
in which the benefits of RVA can unfold. 


18. 
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Policy-makers need to present a stronger 

and more holistic case for the benefits that 
frameworks, skills and qualifications can 
attract, for individuals, employers, and wider 
society. While those benefits typically relate to 
labour market access and longerterm career 
prospects, they also contribute to improved 
physical and mental health, reduced demands 
on criminal justice systems, and lower rates of 
anti-social behaviour. 

Public perception and stakeholder use of 
national frameworks is indeed growing, 
notably in Europe. In recent years, a number 
of countries have subjected their frameworks 
to systemic monitoring and evaluation, aiming 
to sharpen their relevance and better steer 
their implementation. In Montenegro and 
North Macedonia, for example, we see efforts 
to make NOFs more visible to learners and 
employers via dedicated web platforms, and 
of course via indicating NOF and/or ROF levels 
on certificates, as is now the case in most 

EU countries and among EU Neighbourhood 
countries such as Moldova and Georgia. 
Visibility is key to the implementation of the 
South African Qualifications Framework. Its 
lead agency, the South African Qualifications 
Authority, engages with a diverse and extensive 
range of actors from more than 90 organisations 
— public, private and third sector — and has 
already hosted several hundred meetings, 
workshops, and other events. 

There is no shortage of information about skills 
and qualifications. On the contrary, there is a 
‘hosepipe’ supply, which makes filtering and 
selecting relevant material difficult. Skills and 
qualifications information systems, ranging 
from online portals and national agencies to 
qualifications frameworks themselves, both 
provide information, and communicate it 
through various means. Quality assurance and 
standards bodies also participate in this process, 
since part of their purpose is to generate trust 
in the products of skills and qualifications. This 
is coming under renewed scrutiny, as digital 
technology generates new types of credentials 
and qualifications. 

Labour market intelligence systems are also 
being transformed by digital developments, 
such as the collection and analysis of so-called 
big data. Countries and inte-governmental 
agencies, particularly EU institutions, are 
applying a range of new tools to the capture 
and dissemination of the resulting insights. 
This in turn feeds into the complex, but often 
underrepresented area of careers information 
and guidance. By its nature, career guidance has 
always needed a future-oriented focus, and this 
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is even more the case in an era of widespread 
labour market unpredictability. 

With good information, decisions can be made 
about which skills and qualifications to acquire; 
the spotlight then moves on to how to acquire 
them. This involves the art and science of 
teaching and learning — from the identification of 
essential transversal competences to the design 
of curricula at their different levels, macro 

to nano, and their different representations; 
intended, implemented and attained. 

Teachers and trainers are shapers of the 
learning experience, which extends from the 
design of an overall plan to the day-to-day 
management of learning activities. Presenting 
teachers and trainers as both designers of a 
learning process and experts in its delivery 
opens up the prospect of greater integration 
between policy making and practice, and 
stronger support for innovation. Outcomes 
approaches promoted or facilitated by NOFs in 
qualifications are leading to the introduction of 


modular curricula, as in Albania for example. In 
some countries, it is true, these changes do not 
yet apply in all sectors of education and training, 
but they mark significant changes to long- 
established practices. 


. The authors of this volume intentionally 


identified issues and challenges of broad, 

global interest to policy-makers, experts and 
other actors and stakeholders. We see scope 

to increase the profile of national qualifications 
frameworks as communication tools, giving 
them a role in both the delivery and the 
communication of the fundamental benefits that 
people can gain from skills and qualifications. 
National qualifications frameworks look likely to 
remain valuable tools for modern education and 
training systems. The challenge for national and 
international institutions which run qualifications 
frameworks lies in adapting them to the issues 
outlined above; issues that affect their remit and 
that will continue to be generated by ongoing 
change. 
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